Abstract
In a multi-center trial, gadolinium enhanced magnetic resonance angiography (MRA) for diagnosis of acute pulmonary embolism (PE) had a high rate of technically inadequate images. Accordingly, we evaluated the reasons for poor quality MRA of the pulmonary arteries in these patients. We performed a retrospective analysis of the data collected in the PIOPED III study. We assessed the relationship to the proportion of examinations deemed “uninterpretable” by central readers to the clinical centers, MR equipment platform and vendors, degree of vascular opacification in different orders of pulmonary arteries; type, frequency and severity of image artifacts; patient co-morbidities, symptoms and signs; and reader characteristics. Centers, MR equipment vendor and platform, degree of vascular opacification, and motion artifacts influenced the likelihood of central reader determinations that images were “uninterpretable”. Neither the reader nor patient characteristics (age, body mass index, respiratory rate, heart rate) correlated with the likelihood of determining examinations “uninterpretable”. Vascular opacification and motion artifact are the principal factors influencing MRA interpretability. Some centers obtain better images more consistently, but the reasons for differences between centers are unclear.
Keywords: Pulmonary MR angiography, Image quality, Pulmonary embolism, Artifacts
Introduction
Prospective Investigation of Pulmonary Embolism Diagnosis III (PIOPED III) was a prospective, multicenter study designed to assess the sensitivity and specificity of magnetic resonance angiography (MRA) and magnetic resonance angiography combined with magnetic resonance venography (MRV) for the diagnosis of pulmonary embolism (PE).
The PIOPED III study found [1] that MRA, averaged across clinical centers, was “uninterpretable” (technically inadequate) in 92/371 patients (25%). The proportion of “uninterpretable” examinations varied between centers, and ranged from 11 to 51%. Sensitivity of a technically adequate magnetic resonance angiogram ranged from 45 to 100% among the various centers and specificity ranged from 95 to 100%. The most frequent correlates of an “uninterpretable” MRA were poor arterial opacification of segmental or subsegmental branches (67%) and motion artifacts (36%). Severe wrap around artifact was observed in only 4% of patients, and an additional 2% of examinations had severe parallel imaging artifact. Poor image quality was associated with more than one technical shortcoming in 66 of 92 cases (72%) that were rated as “uninterpretable”.
Technically adequate MRA had overall sensitivity 59/76 (78%) and specificity 201/203 (99%) and predictive values that could be clinically useful when combined with pre-test clinical probability [1]. Therefore, among the conclusions of PIOPED III was that further work to improve the consistency of acceptable quality imaging would enhance the clinical value of MRA in patients with suspected PE. It is axiomatic that defining a problem is the first step in solving it. Accordingly, we undertook further analysis in an effort to define what elements of the MRA examination were associated with reader ratings of “interpretable” (diagnostic, adequate) or “uninterpretable” (nondiagnostic, inadequate) technical quality.
Methods
Sites and patients
The institutional review board of each center and by a Data Safety Monitoring Board appointed by the National Heart Lung and Blood Institute approved the protocol and consent forms. All recruited patients gave written informed consent, including for further analysis of their data. In addition to coexisting conditions, PIOPED III recorded whether the patient was an outpatient or inpatient (including intensive care units and long-term care facilities). The methods of identification, recruitment and evaluation of patients have been reported in detail [1].
MRA and MRV methods
Gadolinium enhanced MRA was performed on commercially available 1.5T systems [1] with fast gradient-echo capability (30–40 mT/m max gradient field strength) and slew rate 130–200 mT/msec. At one center (Center 7) a 3.0 T unit (max gradient strength of 40 and 200 mT/msec slew rate) was used in some patients. The protocol was otherwise analogous to that designed for the 1.5 T scanner.
The position of the imaging volume was set to ensure that the descending pulmonary arteries, segmental and proximal subsegmental branches were included in the imaging region.
Imaging was performed using a 3D gradient recalled echo (GRE) sequence in the coronal plane using the following parameters: minimum achievable TR (required to be <6.6 ms), and TE (required to be <2.3 ms), flip angle = 20°–35°, approximately 384 by 288 matrix, 40 cm field of view, bandwidth 380–1,500 Hz/pixel, single acquisition/number of excitations, 3 mm slice thickness (interpolated to 1.5 mm) and at least 44 (88 interpolated) slices or sufficient to cover the anatomy. Parallel imaging with a 6–12 channel phase array coil was used except in large or obese patients in whom phase wrap was deemed likely to occur. With large or obese patients the breath-hold length (no more than 22 s) and field of view were kept constant (40 cm) and the matrix was reduced, thus decreasing the in-plane resolution in the phase-encoding direction.
Imaging time (breath-hold) was approximately 14–22 s. The scan delay was determined using 1–2 ml of contrast agent as a test bolus (348/369 patients whose case files recorded the method used) during a gradient recalled echo sequence performed at 1 image/sec of one slice positioned sagittally through the main pulmonary artery. The scan delay was calculated to place the peak of the infusion enhancement at the center of “k-space” of the pulmonary MRA. In 224/370 patients, centric phase encoding was utilized, while in 146/370 sequential phase encoding was performed. Sites that used bolus tracking were obliged to use centric phase encoding, while sites that did not used sequential or centric encoding according to preference.
Imaging systems produced by four different vendors were employed. The imaging protocol included MRV, but the analyses presented in this study concern only the pulmonary MRA data. Sites were certified as compliant with the technical protocol before patient accrual began. During the study, if the central readers raised quality concerns, study personnel performed site visits and consultations.
Central readings
Two study-certified blinded readers, selected randomly from 13 study–certified readers as previously described [1], interpreted the MRA examinations. Readers were not eligible to read images from their own institution. In addition, readers with a large backlog of assigned but incomplete readings were temporarily removed from the randomization list; accordingly, the randomization was not perfectly balanced. The readers rated each examination as positive for PE, negative for PE, or “uninterpretable”. In addition, the readers rated the degree of vascular opacification as good, fair or poor for three orders of pulmonary arteries: main/lobar; segmental; and subsegmental. Finally, readers identified the type and severity (none, mild, moderate, severe) of common artifacts, including phase aliasing, parallel imaging aliasing and motion artifacts.
Diagnostic criteria for acute pulmonary embolism by magnetic resonance angiography were a partially occlusive intraluminal-filling defect or complete arterial occlusion with termination of the column of contrast material in a meniscus that outlined the trailing edge of the embolus [1]. To be interpreted as negative, an image had to show adequate opacification of subsegmental branches, in the qualitative judgment of the central reader. However, if the central reader saw pulmonary embolism, the examination was defined as technically adequate irrespective of other aspects of image quality.
Analysis and statistical methods
We analyzed the proportion of cases in which both central readers (or the majority of central readers if the first two did not agree) classified the study as “uninterpretable” as related to the MR equipment used and to the clinical center which performed the examination, including the number of cases performed at each center. We quantified the correlates of “uninterpretable” readings from the data available on the central readings: quality of opacification of main/lobar, segmental and subsegmental pulmonary arterial branches; severity of motion artifacts; and severity of aliasing artifacts. We devised a model to assess quality of vascular opacification, as delineated in Table 1. We evaluated whether the breath-hold length required by the exact MR sequence used related to technical quality as assessed by the central readers. Finally, we tested all quality variables for association with “interpretable” or “uninterpretable” classification in a logistic regression model, and we tested significant variables from this model in a final parsimonious model.
Table 1.
A vascular opacification score was created from the opacification variables as shown in the following table
| Main/Lobar | Segmental | Subsegmental | Score |
|---|---|---|---|
| Good | Good | Good | 0 |
| Good | Good | Fair | 1 |
| Good | Fair | Fair | 2 |
| Good | Good | Poor | 2 |
| Fair | Fair | Fair | 3 |
| Good | Fair | Poor | 3 |
| Fair | Fair | Poor | 4 |
| Fair | Poor | Poor | 5 |
| Good | Poor | Poor | 5 |
| Poor | Poor | Poor | 6 |
| Poor | Fair | Poor | 6 |
In addition to these technical factors, we also investigated human factors. We investigated reader effects by analyzing the proportion of “uninterpretable” readings by each central reader. Since assignment of images to central readers was not entirely random, trends in image quality potentially could have resulted from unbalanced reader propensity to label images as “uninterpretable”. Finally, we investigated whether quality differed by patient characteristics including body mass index (BMI), age, Wells Score, heart rate, or respiratory rate, such that centers with a higher proportion of “difficult” patients might have a higher proportion of “uninterpretable” scans.
We estimated sensitivity and specificity of MRA using the methods of Fisher and van Belle [2]. Heterogeneity in quality among clinical centers, vendors, and patient characteristics were examined using chi-square analysis, Fisher’s exact test, and analysis of variance. Correlation between the case volume at each center and the proportion of “uninterpretable” cases was done using Spearman rank correlation. We used the Wilcoxon rank sum test to test for difference in scan time between “interpretable” and “uninterpretable” images, and a non-parametric statistical test, Kruskal–Wallis [3], to test for equality between centers in scan time. We used the Mood test [4] to test for equality in the dispersion of scan time between centers and logistic regression to select the components of image quality that predicted “uninterpretable” classification. We tested model fit using Hosmer and Lemeshow Goodness-of-fit Test. There was no evidence for lack of fit. The 95% Wald confidence intervals are reported.
Finally, we investigated whether the factors that influenced the likelihood of examinations being deemed “uninterpretable” also influenced diagnostic accuracy among “interpretable” examinations. We stratified examinations deemed “interpretable” by the central readers into quartiles according to the vascular opacification score, with or without excluding cases that also had major motion or wrap artifacts, and estimated the sensitivity and specificity in each quartile.
Results
PIOPED III employed imaging systems manufactured by Vendor A in 89 patients (24%), by Vendor B in 149 (40%), by Vendor C in 118 (32%) and by Vendor D in 14 (4%). Most patients (84%) received 0.1 mmol/Kg body weight gadobenate dimeglumine (MultiHance). Gadopentetate dimeglumine, 0.2 mmol/Kg body weight, (Magnevist) was administered to 15% of patients, and the remainder received 0.2 mmol/Kg gadodiamide (Omniscan, GE Health-care, Princeton, NJ) or gadoversetamide (Optimark, Covidien, St. Louis, MO).
There was no statistical difference in the proportion of “uninterpretable” examinations between 1.5T and 3.0T scans (Fisher’s exact Test, P = 0.20).
Correlates of image quality
As previously reported [1], the proportion of “uninterpretable” examinations varied between centers (P < 0.001), ranging from 11 to 51% (Table 2), and the most frequent correlates of an “uninterpretable” MRA were poor arterial opacification of segmental or subsegmental branches (67%) and motion artifacts (36%). The clinical centers were heterogeneous in both the proportion of image artifacts and the frequency of good, fair and poor quality arterial opacification. The quality score for opacification of pulmonary arteries by center is shown in Table 3. The frequency of motion artifacts by center is shown in Table 4.
Table 2.
Number and (%) “uninterpretable” by center and MR vendor from a total of 371 examinations
| Center | MR vendor
|
Total number uninterpretable (% of examinations performed at center “uninterpretable”) | |||
|---|---|---|---|---|---|
| A | B | C | D | ||
| 1 | 7 (47%) | 13 (25%) | 20 (30%) | ||
| 2 | 13 (32%) | 13 (32%) | |||
| 3 | 11 (19%) | 11 (19%) | |||
| 4 | 0 (0%) | 14 (21%) | 14 (18%) | ||
| 5 | 12 (23%) | 12 (23%) | |||
| 6 | 4 (11%) | 4 (11%) | |||
| 7 | 13 (62%) | 5 (36%) | 18 (51%) | ||
| Total | 33 (37%) | 27 (18%) | 27 (23%) | 5 (36%) | 92 |
Table 3.
Vascular opacification score by center
| Opacification score | Center
|
Totala | ||||||
|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | ||
| n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | n | |
| 0 | 12 (17.7) | 13 (31.7) | 23 (39.0) | 16 (20.8) | 20 (37.7) | 11 (29.7) | 2 (5.7) | 97 |
| 1 | 14 (20.6) | 5 (12.2) | 12 (20.3) | 20 (26.0) | 15 (28.3) | 15 (40.5) | 7 (20.0) | 88 |
| 2 | 8 (11.8) | 2 (4.9) | 4 (6.8) | 7 (9.1) | 6 (11.3) | 3 (8.1) | 8 (22.9) | 38 |
| 3 | 14 (20.6) | 9 (22.0) | 12 (20.3) | 21 (27.3) | 7 (13.2) | 5 (13.5) | 6 (17.1) | 74 |
| 4 | 5 (7.35) | 3 (7.3) | 3 (5.1) | 8 (10.4) | 3 (5.7) | 2 (5.4) | 1 (2.9) | 25 |
| 5 | 8 (11.8) | 0 (0.0) | 0 (0.0) | 2 (2.6) | 0 (0.0) | 0 (0.0) | 5 (14.3) | 15 |
| 6 | 7 (10.3) | 9 (22.0) | 5 (8.5) | 3 (3.9) | 2 (3.8) | 1 (2.7) | 6 (17.1) | 33 |
| Total | 68 | 41 | 59 | 77 | 53 | 37 | 35 | 370 |
| % Uninterpretable at center | 30% | 32% | 19% | 18% | 23% | 11% | 51% | |
1 missing value
A score of 0 is equivalent to the best possible opacification, a score of 6 is equivalent to the worst possible opacification
Table 4.
Motion Artifacts by Center
| Motion artifact | Center
|
Totala | ||||||
|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | ||
| None | 2 (3%) | 5 (12%) | 19 (32%) | 18 (23%) | 12 (23%) | 10 (27%) | 11 (32%) | 77 |
| Minor | 48 (71%) | 32 (78%) | 32 (54%) | 43 (56%) | 35 (66%) | 24 (65%) | 16 (47%) | 230 |
| Major | 18 (26%) | 4 (10%) | 8 (14%) | 16 (21%) | 6 (11%) | 3 (8%) | 7 (21%) | 62 |
| Total | 68 | 41 | 59 | 77 | 53 | 37 | 34 | 369 |
2 Missing values
The correlation between the number of cases performed at each center and the proportion of “uninterpretable” cases was −0.37; however, although an inverse correlation might be expected, the relationship was not statistically significant (P = 0.43). The length of the breath-hold required of the patients did vary between centers. Center 1 had a higher average sequence scan time (i.e., breath-hold length) than the other centers, while Center 5 had a lower average scan time (P < 0.001). Centers 1, 2 and 5 also had more variability in scan time (as measured by the interquartile range of scan times) than did Centers 3, 4, 6 and 7 (P < 0.001). In addition, images with no motion artifacts had shorter breath-hold values than images with motion artifacts (P < 0.01). However, there was no statistical difference in breath-hold times between “interpretable” and “uninterpretable” images (P = 0.45); the median breath-hold was 22 s (IQ = 1.0) for “uninterpretable” images and the median breath-hold was 21 s (IQ = 1.0) for “interpretable” images.
The quality of MRA also varied between vendors of the MR imaging devices; (P = 0.008); Table 2 shows the frequency of “uninterpretable” quality by vendor and by center. Vendor B and Vendor C tended to have fewer “uninterpretable” images (on average 18% and 23% “uninterpretable”, respectively). In contrast, Vendor A and Vendor D on average had 37% and 36% “uninterpretable”, respectively. The “uninterpretable” rate for Vendor A ranged, at different centers, from 0 to 62%, while Vendor B (range, 11 to 23%) and Vendor C (range, 21 to 25%) were more consistent across centers. However, we cannot determine definitely whether the differences in quality between centers are due to the vendor of the imaging device; or, to the contrary, that differences in quality between vendors might be due to the users (centers), since PIOPED III was not designed to study such effects and there is confounding between the variables.
We did not find significant differences between readers in the propensity to classify examinations as “uninterpretable”. There were 13 readers who read the 370 scans used in the final analysis of sensitivity and specificity. Excluding three readers who read less than 10 scans, there was no statistical difference in quality assessment between the 10 principal central readers (P = 0.29, Chi-Square).
The various patient characteristics reported in PIOPED III [1] are related to the proportion of non-diagnostic readings in Table 5. None of the patient characteristics that we analyzed quantitatively (BMI, age, Wells Score, heart rate, and respiratory rate) had a statistically significant relationship to technical quality (Table 6). In addition, there is no statistically significant difference according to whether the patient presented while an outpatient or inpatient (outpatient versus all others; P = 0.51; Fisher’s exact test).
Table 5.
Patient characteristics and MRA outcomes (Modified from [1] and reproduced by permission)
| Patients undergoing MRA (N = 371) | MRA “uninterpretable” n (%N) |
|---|---|
| Demographic characteristic | |
| Outpatients (345) | 88 (25.5) |
| Inpatients (23) | 3 (13.0) |
| Female sex (206) | 52 (25.2) |
| Male sex (165) | 40 (24.2) |
| Race | |
| White (252) | 58 (23.0) |
| Black (103) | 32 (31.1) |
| Other (16) | 2 (12.5) |
| Co-Existing conditions | |
| Smoking history (182) | 45 (21) |
| Heart failure (24) | 3 (9) |
| Current asthma (51) | 19 (35) |
| Chronic obstructive pulmonary disease (35) | 11 (28) |
| Current pneumonia (57) | 16 (28.1) |
| Surgery within past 3 months (44) | 10 (22.7) |
| Cancer (60) | 13 (21.7) |
| Symptoms and signs | |
| Dyspnea (118) | 32 (27.1) |
| Pleuritic pain (45) | 16 (35.6) |
| Cough (134) | 42 (31.3) |
| Hemoptysis (16) | 5 (31.3) |
| Tachypnea (≥20 breaths/min) (115) | 28 (24.3) |
| Tachycardia (> 100 beats/min) (54) | 15 (25.9) |
| Wells’ score [17] | |
| Low (226) | 57 (25.2) |
| Moderate (122) | 28 (23.0) |
| High (23) | 7 (30.4) |
Table 6.
Lack of relationship of age, BMI, Wells score [17], heart rate, respiratory rate to ratio of “interpretable” versus “uninterpretable” readings
| Odds Ratio (95% CI) | P–value | |
|---|---|---|
| Age (y) | 1.00 (0.99–1.03) | 0.25 |
| BMI | 0.99 (0.96–1.02) | 0.41 |
| Wells score—low (versus high) | 1.3 (0.51–3.3) | 0.82 |
| Wells score—moderate (versus high) | 1.47 (0.55–3.93) | 0.43 |
| Heart rate (beats/minute) | 1.00 (0.99–1.01) | 0.91 |
| Respiratory rate (breaths/minute) | 1.05 (0.99–1.12) | 0.10 |
Multivariate determinants of “uninterpretable” readings
The results of logistic regression analysis of the quality variables’ association with “interpretable” or “uninterpretable” readings are shown in Table 7. As expected, poorer vascular opacification and more severe motion artifacts were associated with more “uninterpretable” images. The two variables in the final regression were opacification in the segmental and subsegmental pulmonary arteries, and motion artifacts. As the vascular opacification score increases (i.e. becomes poorer), the odds of determining that an image is interpretable decrease. The odds of classifying an image as interpretable increase when there are no artifacts. Significant variables were tested in a final parsimonious model and are shown in the Table 7.
Table 7.
Components of quality by “interpretable” versus “uninterpretable”
| Odds ratio estimates
| ||||
|---|---|---|---|---|
| Effect | Point estimate of odds ratio | 95% wald confidence limits | P val | |
| Score* (0 versus 6) | 229.668 | 49.788 | > 999.999 | < 0.0001 |
| Score (1 versus 6) | 112.364 | 27.386 | 783.241 | 0.0001 |
| Score (2 versus 6) | 90.542 | 19.785 | 683.552 | 0.0059 |
| Score (3 versus 6) | 54.069 | 13.914 | 363.175 | 0.0293 |
| Score (4 versus 6) | 53.093 | 11.290 | 404.250 | 0.1287 |
| Score (5 versus 6) | 1.288 | 0.056 | 14.924 | 0.0009 |
| Motion artifacts (none versus major) | 0.222 | 0.067 | 0.662 | 0.0093 |
| Motion artifacts (minor versus major) | 0.271 | 0.127 | 0.579 | 0.0007 |
As the vascular opacification score increases (i.e. opacification becomes poorer), the odds of determining that an image is interpretable decrease The odds of classifying an image as interpretable increase when there are no motion artifacts
The different artifacts were correlated. The quality score for opacification was positively correlated with worse motion artifacts (Spearman r = 0.183 P = 0.0004). Worse motion artifacts were also positively correlated with wrap and parallel imaging artifacts (Spearman r = 0.208, P < 0.0001 and r = 0.149, P = 0.004, respectively). Wrap artifact was correlated with parallel imaging artifact (Spearman r = 0.51, P < 0.0001). As might be expected, poorer opacification on the quality score was positively correlated with the reader’s subjective impression that the scan timing was early or late relative to contrast injection (Spearman r = 0.484, P < 0.0001).
Diagnostic accuracy and quality
There was no clear relationship between the proportion of technically inadequate examinations at a particular center and the sensitivity or specificity for detecting PE of those examinations that were deemed technically adequate. All centers had specificity of 100% except for one center with 95% specificity (that center had a proportion of technically inadequate examinations that was fifth highest of seven centers, but not the worst of the group). The centers with the lowest sensitivity ranked fourth and fifth of seven centers for proportion of technically adequate examinations, but the lowest ranked two centers had sensitivity as good as the highest. Among interpretable examinations, there was no obvious relationship between quality and diagnostic accuracy. As shown in Table 8, when the examinations deemed “interpretable” by the central readers were stratified into quartiles according to the vascular opacification score, there was no significant difference of sensitivity or specificity between the quartiles. This lack of relationship also pertained when examinations with major motion and wrap artifacts were excluded from the analysis.
Table 8.
Sensitivity and specificity of “interpretable” MRA according to quality of vascular opacification
| Quartile of vascular opacification score (min, max) | Reference PE and MR PE | Reference PE and MR No PE | Reference no PE and MR no PE | Reference no PE and MR PE | Sensitivity | Specificity |
|---|---|---|---|---|---|---|
| n | n | n | n | P (95% CI) | P (95% CI) | |
| Sensitivity and specificity by quartile of opacification score | ||||||
| 1 (0,0) | 21 | 3 | 33 | 1 | 0.88 (0.68–0.97) | 0.97 (0.85–1.00) |
| 2 (1,1) | 14 | 1 | 33 | 1 | 0.93 (0.68–1.00) | 0.97 (0.85–1.00) |
| 3 (2,3) | 22 | 2 | 28 | 1 | 0.92 (0.73–0.99) | 0.97 (0.82–1.00) |
| 4 (4,6) | 8 | 0 | 7 | 1 | 1.00 (0.63–1.00) | 0.88 (0.47–1.00) |
| Sensitivity and specificity by quartile of opacification score with MRAs with major motion artifacts and major wrap artifacts removed | ||||||
| 1 (0,0) | 20 | 3 | 32 | 1 | 0.87 (0.66–0.97) | 0.97 (0.84–1.00) |
| 2 (1,1) | 13 | 1 | 32 | 1 | 0.93 (0.66–1.00) | 0.97 (0.84–1.00) |
| 3 (2,3) | 20 | 1 | 21 | 1 | 0.95 (0.76–1.00) | 0.95 (0.77–1.00) |
| 4 (4,6) | 5 | 0 | 5 | 0 | 1.00 (0.48–1.00) | 0.83 (0.36–1.00) |
Discussion
The PIOPED III study found [1] that MRA, averaged across clinical centers, was “uninterpretable” (technically inadequate) in 92/371 patients (25%). The proportion of “uninterpretable” examinations varied between centers, and ranged from 11 to 51%. Accordingly, local ability to obtain adequate image quality is an important variable in assessing whether MRA may be a clinically useful test for PE at a particular site. Technically adequate MRA had sensitivity 59/76 (78%) and specificity 201/203 (99%) and predictive values that could be clinically useful when combined with pre-test clinical probability [1]. We are not aware of a generally applicable standard for the required prevalence of a “technically adequate” test. However, since none of the PIOPED III centers had “uninterpretable” MRA rates even close to the ~6% nondiagnostic rates obtained with CT angiography (CTA) in PIOPED II, we believe it is apparent that the technical quality of MRA needs to be improved. Accordingly, improving the rate of diagnostic-quality MRA images should be a high priority for even the better performing sites in PIOPED III. Therefore, among the conclusions of PIOPED III was that further work to improve the consistency of acceptable quality imaging would enhance the clinical value of MRA in patients with suspected PE. Accordingly, we undertook further analysis in an effort to define what elements of the MRA examination were associated with reader ratings of “interpretable” (diagnostic, adequate) or “uninterpretable” (nondiagnostic, inadequate) technical quality.
The most frequent correlates of an “uninterpretable” MRA were poor arterial opacification of segmental or subsegmental branches (67%) and motion artifacts (36%). Severe wrap around artifact was observed in only 4% of patients, and an additional 2% of examinations had severe parallel imaging artifact. Poor image quality was associated with more than one technical shortcoming in 66 of 92 cases (72%) that were rated as “uninterpretable”.
Both the equipment used and the manner in which it was deployed in clinical examinations at the different clinical centers in PIOPED III seemed to be significant. However, not all centers had MR scanners from all of the vendors (Table 1). Therefore, there were numerous empty cells in the center by vendor matrix and we were unable to determine to what extent the interaction terms were significant (or possibly even dominant). If equipment factors primarily explained differences in quality, then we would expect the same rate of “uninterpretable” images between centers for a given vendor, but this was not observed consistently. This variation could possibly be due to vendor platform variation (gradient performance, software version variations, and so forth) among sites, different levels of technologist training, or different levels of radiologist expertise and attention to the protocol. Inquiries to the clinical centers did not reveal any obvious explanations of these types, but we nevertheless believe that such factors are probably very important. The rates of “uninterpretable” images by center for Vendor A machines were highly variable, while the rates for Vendor B and Vendor C machines were much more consistent; we can only speculate as to the reasons for this difference. Again, because of empty cells we cannot do a regression of the interaction terms. We explored a subgroup analysis of centers that used more than one vendor’s device, but it limited the comparison to two centers (Center 1 and Center 4) and two vendors, and in addition a difference in field strength confounded the comparison. We evaluated differences in quality by center stratified by vendor; but this requires the unverified assumption of a significant interaction between center and vendor. That analysis does again suggest that there are differences in quality between centers using the Vendor A machines (Vendor A, P = 0.033; Vendor B, P = 0.355; Vendor C, P = 0.627). It would be desirable to be able to ascertain more definitely the relationship between the capabilities of the imaging equipment and the expertise of those using it, but since the data collection and the study design of PIOPED III were not crafted to evaluate this question, the data available do not permit us to do so.
Technical factors do appear to be more significant than patient factors or reader effects in producing good or poor quality images. This is in contrast with a recent study of CTA for PE, in which patient-related factors were considered by the interpreting radiologist to limit the interpretation of CTA in 72% of patients [5]. Vascular opacification and motion artifacts were the dominant phenomena leading to images being classified “uninterpretable”, with poor vascular opacification being the most commonly noted shortcoming associated with poor quality images.
Accordingly, in efforts to reduce the number of technically inadequate examinations, improving vascular opacification is likely to be the most promising approach. One avenue for attempting this might be through use of intravascular magnetic resonance contrast agents [6], which might reduce substantially the number of “uninterpretable” studies due to difficulties in timing the bolus, and allow repeat imaging (for example if the patient moved or breathed during the initial imaging). That the choice of contrast agent makes a difference is shown in a recent analysis [7] of PIOPED III patients who received different gadolinium agents, in which statistically significant differences in pulmonary SNR (P = 0.01) and CNR (P = 0.008) were found between gadobenate dimeglumine and gadopentetate dimeglumine, with SNR and CNR higher on examinations using gadobenate dimeglumine. Subjective quality at each vessel order also was significantly better for gadobenate dimeglumine (P < 0.0001). Because of the presumed importance of showing the majority of elastic (>1 mm diameter) pulmonary artery branches, in all PIOPED III MRA examinations that were interpreted as negative, the central reader made a qualitative judgment that pulmonary artery opacification was adequate through the subsegmental branches [1]. Several prior studies have defined an adequate quality magnetic resonance angiography as adequate opacification through segmental vessels [8–11]. Whether it is important to diagnose pulmonary embolism in sub-segmental branches has been debated [12]. Only 1 of 102 (1%) diagnosed by computed tomographic angiography (CTA) had pulmonary embolism limited to subsegmental branches [1]. Pulmonary embolism limited to subsegmental branches was shown in 22 of 375 (6%) in PIOPED I [13] and in 8 of 175 (5%) in PIOPED II [14]. In addition, it is probably not possible to diagnose subsegmental PE accurately with any currently available method. With mostly 4-detector CTA, the positive predictive value of apparent pulmonary embolism limited to subsegmental branches was only 25% (2 of 8) [14] and with conventional pulmonary angiography, readers could agree on only 2 of 15 cases of pulmonary embolism limited to subsegmental branches [15]. Nevertheless, our regression analysis demonstrates that readers consider the visualization of subsegmental arteries in determining whether an MRA is “uninterpretable”.
There was no clear relationship between the proportion of technically inadequate examinations at a particular center and the sensitivity or specificity for detecting PE of those examinations that were deemed technically adequate. Among interpretable examinations, there was no obvious relationship between quality and diagnostic accuracy. One reasonable explanation for the high association of vascular opacification and artifact and the likelihood of examinations being “uninterpretable”, but lack of relationship between apparently higher quality and lower quality examinations by these measures within the group of “interpretable” MRA is a threshold effect—that is, beyond a certain level of opacification and lack of artifact, further improvements in technical quality do not affect diagnostic accuracy. Since the distribution of “easy” (e.g., large central emboli) and “difficult” (e.g., small peripheral emboli) cases would not be expected to follow the distribution of technical proficiency, perhaps correlations to quality by itself would not be anticipated once quality reaches a threshold level.
Efforts to improve the proportion of diagnostic-quality pulmonary MRA examinations should be directed to motion reduction techniques and methods to facilitate the optimal signal intensity of the pulmonary arteries during image acquisition (especially during acquisition of the central k-space views). Further improvement in bolus tracking and scan synchronization using conventional approaches are possible, of course. Intravascular contrast agents may reduce the precision needed for timing image acquisition with contrast administration, at the cost of opacifying irrelevant vessels such as pulmonary veins. Non-contrast-enhanced MRA imaging methods such as steady-state free precession techniques coupled to respiratory navigators, which allow free-breathing during imaging [16] have not been reported for PE but might also be useful avenues for investigation.
In summary, technical characteristics of pulmonary MRA test performance influenced the likelihood of reader determinations that images were “uninterpretable”, but neither the reader nor patient characteristics correlated with the likelihood of determining examinations “uninterpretable”.
Acknowledgments
This study was supported by Grants HL081593, HL177150, HL077149, HL077151, HL077154, HL081594, HL077358, HL077155, and HL077153 from the U.S. Department of Health and Human Services, Public Health Services, National Heart, Lung, and Blood Institute, Bethesda, Maryland.
Footnotes
Conflict of interest Dr. Woodard received research support from GE Healthcare and Siemens Medical Systems.
Contributor Information
H. Dirk Sostman, Office of the Dean and Department of Radiology, Weill Cornell Medical College and The Methodist Hospital, Houston, Texas.
Kathleen A. Jablonski, The Biostatistics Center, George Washington University, Rockville, MD, USA
Pamela K. Woodard, Email: woodardp@mir.wustl.edu, Department of Radiology, Mallinckrodt Institute of Radiology, Washington University School of Medicine, 510 S. Kingshighway Blvd., St. Louis, MO, USA
Paul D. Stein, Department of Internal Medicine, College of Osteopathic Medicine, Michigan State University, East Lansing, MI, USA
David P. Naidich, Department of Radiology, New York University Medical Center, New York City, NY, USA
Thomas L. Chenevert, Department of Radiology, University of Michigan, Ann Arbor, MI, USA
John G. Weg, Department of Medicine, University of Michigan, Ann Arbor, MI, USA
Charles A. Hales, Department of Medicine, Massachusetts General Hospital, and Harvard Medical School, Boston, MA, USA
Russell D. Hull, Department of Medicine, University of Calgary, Calgary, AB, Canada
Lawrence R. Goodman, Department of Radiology, Medical College of Wisconsin, Milwaukee, WI, USA
Victor F. Tapson, Department of Medicine, Duke University Medical Center, Durham, NC, USA
References
- 1.Stein PD, Chenevert TL, Fowler SE, et al. for the PIOPED III Investigators. Gadolinium enhanced magnetic resonance angiography for pulmonary embolism: a multicenter prospective study (PIOPED III) Ann Int Med. 2010;152:434–443. doi: 10.1059/0003-4819-152-7-201004060-00008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Fisher L, van Belle G. Biostatistics: a methodology for the health sciences. New York: Wiley; 1993. p. 206. [Google Scholar]
- 3.Gibbons JD, Chakraborti S. Nonparametric statistical inference. 3. Marcel Dekker, Inc; New York: 1992. p. 295. [Google Scholar]
- 4.Gibbons JD, Chakraborti S. Nonparametric statistical inference. 3. Marcel Dekker, Inc; New York: 1992. p. 264. [Google Scholar]
- 5.Abujudeh HH, Kaewlai R, Farsad K, Orr E, Gilman M, Shepard JO. Computed tomography pulmonary angiography: an assessment of the radiology report. Acad Radiol. 2009;16:1309–1315. doi: 10.1016/j.acra.2009.06.012. [DOI] [PubMed] [Google Scholar]
- 6.Hadizadeh DR, Gieseke J, Lohmaier SH, et al. Peripheral MR angiography with blood pool contrast agent: prospective intraindividual comparative study of high-spatial-resolution steady-state MR angiography versus standard-resolution first-pass MR angiography and DSA. Radiology. 2008;249:701–711. doi: 10.1148/radiol.2492072033. [DOI] [PubMed] [Google Scholar]
- 7.Woodard PK, Chenevert TL, Sostman HD, Jablonski KA, Stein PD, Goodman LR, Londy FJ, Narra V, Hales CA, Hull RD, Tapson VF, Weg JG. Signal quality of single dose gadobenate dimeglumine pulmonary MRA examinations exceeds quality of MRA performed with double dose gadopentetate dimeglumine. Int J Cardiovasc Imaging. 2011 doi: 10.1007/s10554-011-9821-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Loubeyre P, Revel D, Douek P, et al. Dynamic contrast-enhanced MR angiography of pulmonary embolism: comparison with pulmonary angiography. AJR Am J Roentgenol. 1994;162:1035–1039. doi: 10.2214/ajr.162.5.8165977. [DOI] [PubMed] [Google Scholar]
- 9.Ohno Y, Higashino T, Takenaka D, et al. MR angiography with sensitivity encoding (SENSE) for suspected pulmonary embolism: comparison with MDCT and ventilation-perfusion scintigraphy. AJR Am J Roentgenol. 2004;183:91–98. doi: 10.2214/ajr.183.1.1830091. [DOI] [PubMed] [Google Scholar]
- 10.Ersoy H, Goldhaber SZ, Cai T, et al. Time-resolved MR angiography: a primary screening examination of patients with suspected pulmonary embolism and contra-indications to administration of iodinated contrast material. AJR Am J Roentgenol. 2007;188:1246–1254. doi: 10.2214/AJR.06.0901. [DOI] [PubMed] [Google Scholar]
- 11.Blum A, Bellou A, Guillemin F, Douek P, Laprevope-Heully MC, Wahl D GENEPI study group. Performance of magnetic resonance angiography in suspected acute pulmonary embolism. Thromb Haemost. 2005;93:503–511. doi: 10.1160/TH04-08-0495. [DOI] [PubMed] [Google Scholar]
- 12.Goodman LR. Small pulmonary emboli: what do we know? (Editorial) Radiology. 2005;234:654–658. doi: 10.1148/radiol.2343041326. [DOI] [PubMed] [Google Scholar]
- 13.Stein PD, Henry JW. Prevalence of acute pulmonary embolism in central and subsegmental pulmonary arteries and relation to probability interpretation of ventilation/perfusion lung scans. Chest. 1997;11:1246–1248. doi: 10.1378/chest.111.5.1246. [DOI] [PubMed] [Google Scholar]
- 14.Stein PD, Fowler SE, Goodman LR, et al. for the PIOPED II Investigators. Multidetector computed tomography for acute pulmonary embolism. N Eng J Med. 2006;354:2317–2327. doi: 10.1056/NEJMoa052367. [DOI] [PubMed] [Google Scholar]
- 15.Quinn MF, Lundell CJ, Klotz TA, et al. Reliability of selective pulmonary arteriography in the diagnosis of pulmonary embolism. Am J Roent. 1987;149:469–471. doi: 10.2214/ajr.149.3.469. [DOI] [PubMed] [Google Scholar]
- 16.Krishnam MS, Tomasian A, Deshpande V, et al. Noncontrast 3D steady-state free-precession magnetic resonance angiography of the whole chest using nonselective radiofrequency excitation over a large field of view: comparison with single-phase 3D contrast-enhanced magnetic resonance angiography. Invest Radiol. 2008;43:411–420. doi: 10.1097/RLI.0b013e3181690179. [DOI] [PubMed] [Google Scholar]
- 17.Wells PS, Anderson DR, Rodger M, et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and d-dimer. Ann Intern Med. 2001;135:98–107. doi: 10.7326/0003-4819-135-2-200107170-00010. [DOI] [PubMed] [Google Scholar]