Abstract
Purpose
To determine if second-opinion review of gynaecologic oncologic (GynOnc) magnetic resonance imaging (MRI) by sub-specialized radiologists impacts patient care.
Methods
Four hundred and sixty-nine consecutive second-opinion MRI interpretations rendered by GynOnc radiologists were retrospectively compared to the initial outside reports. Two gynaecologic surgeons, blinded to the reports’ origins, reviewed all cases with discrepancies between initial and second-opinion MRI reports and recorded whether these discrepancies would have led to a change in patient management defined as a change in treatment approach, counselling, or referral. Histopathology or minimum 6-month imaging follow-up were used to establish the diagnosis.
Results
Second-opinion review of GynOnc MRIs would theoretically have affected management in 94/469 (20%) and 101/469 (21.5%) patients for surgeons 1 and 2, respectively. Specifically, second-opinion review would have theoretically altered treatment approach in 71/469 (15.1%) and 60/469 (12.8%) patients for surgeons 1 and 2, respectively. According to surgeons 1 and 2, these treatment changes would have prevented unnecessary surgery in 35 (7.5%) and 31 (6.6%) patients, respectively, and changed surgical procedure type/extent in 19 (4.1%) and 12 (2.5%) patients, respectively. Second-opinion interpretations were correct in 103 (83%) of 124 cases with clinically relevant discrepancies between initial and second-opinion reports.
Conclusions
Expert second-opinion review of GynOnc MRI influences patient care.
Keywords: magnetic resonance imaging, gynaecologic oncologic imaging, second opinion, subspecialty radiologists, cancer
INTRODUCTION
The need to simultaneously improve the quality of health care (patient outcomes) and reduce health care costs has been widely recognized nationally and internationally [1–4]. In the United States, a value-based health care delivery system – where payments are based on outcomes rather than the volume of services rendered – is increasingly being pursued as a solution, thanks in part to the passage of the Affordable Care Act [5; 6].
An important feature of a value-based system is the organization of health care delivery around integrated practice units, in which physicians with relevant expertise work jointly to provide coordinated, evidence-based care for patients with a single medical condition. Today, many oncology centres, including our tertiary care cancer centre, rely on multidisciplinary disease management teams (DMT) who care for patients with specific types of cancer. Because precise diagnosis and assessment of disease extent are considered essential for determining an appropriate treatment plan, each DMT, in addition to specialists in surgery, medical oncology, radiation oncology, and pathology, includes radiologists with relevant, focused expertise. Before DMT meetings at our institution, imaging studies obtained elsewhere are often re-interpreted by such sub-specialized radiologists and an official second-opinion report is issued.
Rates of disagreement between initial and second-opinion imaging interpretations have been examined in a number of studies, in fields such as oncologic imaging, abdominal imaging, neuroradiology, paediatric imaging, and emergency radiology [7–18]. Some of these studies emphasized the frequency of minor and major discrepancies rather than their impact on patient care as determined by referring clinicians. The aim of this study was to determine the added value of second-opinion review of gynaecologic oncologic (GynOnc) MRI examinations by GynOnc radiologists, as assessed by treating physicians. We decided to focus on GynOnc MRI because of the limited prior literature in this area and the fact that GYN MRI scans are commonly interpreted by general body radiologists, who may have limited subspecialty training in gynaecologic MRI [19; 20].
MATERIALS AND METHODS
Our institutional review board approved and issued a waiver of informed consent for this retrospective study, which was compliant with the Health Insurance Portability and Accountability Act.
Eligibility
A retrospective search of our institutional clinical database was performed to identify all patients fulfilling the following inclusion criteria:(i) consecutive submitted GynOnc MRI performed and initially interpreted at an outside institution between January 1st 2008 and August 1st 2013; (ii) second-opinion GynOnc MRI interpretation documented in an official report issued by one of four sub-specialized gynaecologic radiologists at our institution; and (iii) histopathology or ≥ 6-month follow-up imaging after the MRI. This search yielded 525 outside GynOnc MRI studies with both initial reports and second-opinion interpretations. To ensure that second-opinion review did not benefit from extra clinical information, we excluded 56 MRI studies obtained in patients who had biopsy and/or surgery (45 patients) or additional diagnostic imaging test(s) (11 patients) performed between the initial report and second-opinion interpretation. The final sample consisted of 469 consecutive submitted GynOnc MRI studies obtained outside our institution, each of which had been reinterpreted by one of four gynaecologic oncologic radiologists with at least 5 years of sub-specialty imaging experience gained by actively participating in the gynaecologic oncologic tumour board.
Data Analysis and Interpretation
Two radiologists (with 5 and 7 years of experience in diagnostic imaging), who did not participate in the second-opinion interpretations, examined the initial and second-opinion interpretations of each MRI and, in consensus, divided all the MRI studies into two groups: i) NO disagreement and ii) ANY disagreement between initial and second-opinion interpretations.
Two board-certified gynaecologic oncologic surgeons, each with 20 years of experience, independently reviewed initial and second-opinion reports for all cases in which the two reports contained any disagreements. The surgeons were provided with relevant clinical history but were blinded to the origin of each MRI report (i.e., each surgeon was presented with de-identified written summary of key imaging findings for each MRI report as illustrated in Figure 1. The two radiologists described above created these summaries. The surgeons were blinded to the origin of each MRI report throughout the study). For every pair of MRI reports, each surgeon recorded whether differences between the reports were clinically important, i.e. would theoretically have led to a change in patient management (Figure 1). A change in patient management was defined as an alteration in any of the following: i) treatment approach; ii) patient counselling; iii) patient referral (tertiary care centre versus community-based practice). In instances where the difference between the initial and second-opinion reports would have led to a modification in treatment approach, each surgeon was asked to record their preferred treatment strategy and any additional required tests, basing their judgment on each MRI report and the relevant clinical history provided.
When clinically relevant discrepancies were identified between initial and second-opinion interpretations, the precision of the second-opinion interpretation was evaluated by using histopathology (in 103/124 cases [83.1%]) or by using minimum 6-month imaging follow-up (in 21/124 cases [16.9%]) as the reference standard.
The number of MRI examinations that were considered limited due to their image quality by the sub-specialized radiologists was noted. The reasons for such assessment were obtained from the second-opinion report and classified as follows: 1) artefacts (for example, motion-induced image blurring), 2) absence of key pulse sequence(s) (for instance, the lack of oblique axial T2-weighted images for the assessment of parametria in studies acquired for the initial staging of cervical cancer or the absence of multiphase contrast-enhanced sequences in the studies obtained for the initial staging of the endometrial cancer), 3) suboptimal imaging technique (for example, low magnet strength, large field-of-view).
Statistical Analysis
Confidence intervals (CI) were estimated using the Wilson score interval with continuity correction [21]. The programming language R (version 3.1.1; R Foundation for Statistical Computing, Vienna, Austria) was used to perform all statistical computations.
RESULTS
The median patient age was 51 years (range: 18–90 years). Of 469 women included in the study, 130 women had a known diagnosis of cancer; 123 of these 130 women had gynaecologic malignancies and 7 had either non-gynaecologic cancers or cancers of unknown primary (Table 1). The GynOnc MRI examinations were obtained for a variety of clinical indications, summarized in Table 2 and Figure 2.
Table 1.
Patients | N =469, (%) |
---|---|
Age, years | |
Median | 51 |
Range | 18 – 90 |
No established cancer diagnosis | 339 (72.3%) |
Known cancer | 130 (27.7%) |
• Gynaecologic malignancies | 123 (26.2%) |
○ Endometrial carcinoma (CA) | 36 |
○ Cervical CA | 41 |
○ Adenocarcinoma of either endometrial or cervical origin | 3 |
○ Ovarian CA | 19 |
○ Vaginal or vulvar squamous cell carcinoma (SCC) | 7 |
○ Uterine sarcomas | 7 |
○ Gestational Trophoblastic Diseases | 6 |
○ Vaginal or vulvar melanoma | 2 |
○ Vaginal leiomyosarcoma | 1 |
○ Spindle cell sarcoma of the adnexa | 1 |
• Non-gynaecologic malignancies | 7 (1.5%) |
○ Breast CA | 2 |
○ Adrenal cell carcinoma with vaginal metastasis | 1 |
○ Cancer of unknown primary with peritoneal carcinomatosis | 3 |
○ Retroperitoneal leiomyosarcoma | 1 |
Table 2.
Organ | Indications for MRI | N | Total | % |
---|---|---|---|---|
Uterus | Evaluation of symptomatic or enlarging leiomyomas | 34 | 96 | 20.5 |
Staging of newly diagnosed endometrial CA | 25 | |||
Suspected recurrent endometrial CA | 9 | |||
Recurrent endometrial cancer; restaging | 2 | |||
Evaluation of endometrial thickening or mass seen on prior US | 10 | |||
Staging of uterine leiomyosarcoma | 1 | |||
Suspected recurrent uterine sarcoma | 6 | |||
Staging of gestational trophoblastic disease | 6 | |||
Suspected uterine adenomyosis | 2 | |||
Indeterminate uterine mass on US | 1 | |||
Cervix | Staging of newly diagnosed carcinoma of the cervix | 37 | 54 | 11.5 |
Suspected recurrent cervical CA | 4 | |||
Evaluation of cervical mass seenon US or CT | 10 | |||
Abnormal pap smear; cervical vs. endometrial origin of adenocarcinoma | 3 | |||
Adnexa | Characterization of indeterminate ovarian mass seen on US or CT | 154 | 186 | 39.6 |
Elevated CA-125 | 9 | |||
Staging of ovarian cancer | 7 | |||
Suspected recurrent ovarian cancer | 10 | |||
Recurrent ovarian cancer; restaging | 2 | |||
Indeterminate ovarian lesion in patients with known breast CA | 2 | |||
Suspected hydrosalpinx on CT | 1 | |||
Suspected recurrent spindle cell sarcoma of the adnexa | 1 | |||
Vagina/Vulva | Staging of newly diagnosed vaginal or vulvar SCC | 5 | 12 | 2.6 |
Known recurrent vulvar SCC; restaging | 1 | |||
Suspected recurrent vaginal SCC | 1 | |||
Staging of vaginal melanoma | 1 | |||
Suspected recurrent vulvar melanoma | 1 | |||
Staging of newly diagnosed vaginal leiomyosarcoma | 1 | |||
Metastasis to the vagina from the known adrenocortical CA | 1 | |||
Indeterminate vulvar mass | 1 | |||
Other | Pelvic mass seen on US, further evaluation to determine uterine versus ovarian origin | 66 | 121 | 25.8 |
Weight loss, pelvic pain, increasing abdominal girth or abdominal distention | 34 | |||
Menorrhagia, vaginal bleeding, or postmenopausal bleeding | 9 | |||
Cancer of unknown primary with peritoneal carcinomatosis detected on CT | 3 | |||
Pelvic lymphadenopathy of uncertain origin | 3 | |||
Indeterminate pelvic cystic lesion on US (e.g. possible urachal cyst) | 2 | |||
Endometriosis | 2 | |||
Suspected recurrent retroperitoneal leiomyosarcoma | 1 | |||
Indeterminate mass at the surgical port site | 1 | |||
Total | 469 | 100 |
GynOnc MRI studies were submitted from 178 different institutions. Four hundred and twenty-three (90.2%) of the 469 MRI studies were first interpreted by radiologists at private community hospitals or outpatient radiology facilities, and 46 (9.8%) were initially read by radiologists at academic tertiary care centres; 438 (93%) included contrast-enhanced sequences. The median time interval between initial and second-opinion interpretations was 19 days (range: 1–181 days).
Second-opinion review found 82 (17.5%) of 469 MRI studies to be of limited imaging quality (Figure 3); repeat imaging was recommended in 25 (30.5%) of 82 MRI examinations, and 7 MRIs were immediately repeated to compensate for the technical limitation of the initial study. The reasons for this limited imaging quality assessment are summarized in Table 3.
Table 3.
Limitations | Number of patients | % (95% CI) |
---|---|---|
Artefacts | 4/82 | 5% (95%CI: 1.6–12.7) |
Absence of key pulse sequence(s) | 14/82 | 17% (95%CI: 10.0–27.3) |
Suboptimal imaging technique | 20/82 | 24% (95% CI: 15.9–35.3) |
Artefacts + absence of key pulse sequence(s) | 10/82 | 12% (95% CI: 6.3–21.7) |
Artefacts + suboptimal imaging technique | 22/82 | 27% (95% CI: 17.9–37.9) |
Suboptimal imaging technique + absence of key pulse sequence(s) | 8/82 | 10% (95%CI: 4.6–18.8) |
Artefacts + absence of key pulse sequence(s) + suboptimal imaging technique | 4/82 | 5% (95%CI: 1.6–12.7) |
MRI Report Review
Preliminary Review by Radiologists
The preliminary review by two radiologists found that for 288/469 MRI studies (61.4%; 95%CI:56.8–65.8), there were no disagreements between the initial and second-opinion interpretations, while for 181/469 (38.6%; 95%CI:34.2–43.2), there was some disagreement (either clinically unimportant or important). The discrepancies concerned the number and locations of lesions detected (for example, presence of pelvic lymphadenopathy or peritoneal carcinomatosis) in 37/469 (7.9%; 95%CI:5.7–10.8), interpretation of findings (for example, determination of the origin of an adnexal mass) in 124/469 (26.4%; 95%CI:22.6–30.7) or both in 20/469 (4.3%; 95%CI:2.7–6.6) patients (Figure 3).
Review by Surgeons – Combined Results
For 124 (68.5%; 95%CI:61.1–75.1) of the 181 cases with discrepant MRI reports, the discrepancies were deemed clinically important by at least one of the gynaecologic oncologic surgeons. In other words, discrepancies between initial and second-opinion interpretations would theoretically have affected patient care in up to 124/469 (26.4%; 95%CI: 22.6–30.7) patients. Second-opinion review removed the suspicion of gynaecologic malignancy raised by the initial report for 46/124 (37%; 95%CI:28.7–46.3) patients, suggested a new diagnosis of cancer omitted from the initial report for 12/124 (10%; 95%CI:5.3–16.6) patients, upstaged or downstaged disease for 18/124 (14%; 95%CI:9.1–22.2) patients, and provided a more specific diagnosis (for example, epithelial ovarian neoplasm versus ovarian mass) in 48/124 (39%; 95%CI:30.2–47.9) patients.
Comparison to histopathology or follow-up imaging showed that the second-opinion interpretation was correct for 103 (83%; 95%CI:75.0–89.0) of the 124 MRI studies for which at least one surgeon identified clinically important discrepancies between initial and second-opinion interpretations. Neither initial nor second-opinion report was correct in 12/124 cases (9.7%; 95% CI: 5.3–16.6). Initial reports were correct in only 9/124 (7.3%; 95% CI: 3.6–13.7) of cases with clinically important disagreements.
Review by Surgeons – Individual Results
Detailed summaries of the results for each surgeon are provided in Tables 4 and 5. Briefly, second-opinion review of GynOnc MRI would have theoretically affected patient care in 94/469 (20%; 95% CI:16.5–24.0) patients for surgeon 1 and 101/469 (21.5%; 95%CI:17.9–25.6) patients for surgeon 2. The treatment approach would have been altered in 71/469 (15.1%; 95%CI:12.1–18.8) and 60/469 (12.8; 95%CI:10.0–16.2) patients for surgeons 1 and 2, respectively. According to surgeons 1 and 2, these treatment changes would have prevented unnecessary surgery in 35 (7.5%) and 31(6.6%) patients, respectively, and would have altered surgical procedure type/extent in 19 (4.1%) and12 (2.5%) patients, respectively. Additionally, discrepancies between initial and second-opinion interpretations would have led to changes in counselling for 92/469 (19.6%; 95%CI:16.2–23.6) patients for surgeon 1 and 101/469 (21.5%; 95%CI:18.0–25.6) patients for surgeon 2.
Table 4.
SURGEON 1 | SURGEON 2 |
---|---|
Number of patients (%) | |
Change in Management | |
94/469 (20.0%) (95%CI:16.6–24.0) |
101/469 (21.5%) (95%CI:18.0–25.6) |
Change in Treatment Approach | |
71/469 (15.1%) (95%CI: 12.1–18.8) |
60/469 (12.8%) (95%CI: 10.0–16.2) |
Change in Patient Counselling | |
92/469 (19.6%) (95%CI: 16.2–23.6) |
101/469 (21.5%) (95%CI:18.0–25.6) |
Change in Patient Referral | |
50/469 (10.7%) (95%CI:8.1–13.9) |
53/469 (11.3%) (95%CI:8.6–14.6) |
Table 5.
Change in Treatment Approach (Outside reports → 2nd-opinion interpretation) | Number of patients | % |
---|---|---|
SURGEON 1 | ||
Surgical → Non-surgical approach | 35/469 | 7.5% (95%CI:5.3–10.3) |
Change in surgical procedure type/extent | 19/469 | 4.1% (95%CI:2.5–6.4) |
Non-surgical → Surgical approach | 11/469 | 2.3% (95%CI:1.2–4.3) |
Biopsy → Observation or follow-up with imaging | 4/469 | 0.85% (95%CI:0.3–2.3) |
Follow-up with imaging → Colonoscopy | 1/469 | 0.2% (95%CI:5.3–10.3) |
Observation → Follow-up with imaging | 1/469 | 0.2% (95%CI:0.01–1.37) |
SURGEON 2 | ||
Surgical → Non-surgical approach | 31/469 | 6.6% (95%CI:4.6–9.4) |
Change in surgical procedure type/extent | 12/469 | 2.5% (95%CI:1.4–4.6) |
Non-surgical → Surgical approach | 9/469 | 1.9 % (95%CI:0.9–3.7) |
Biopsy or Dilatation and Curettage → Surgical approach | 3/469 | 0.6% (95%CI:0.2–2.0) |
Biopsy → Observation | 2/469 | 0.4% (95%CI:0.07–1.70) |
Follow-up with imaging → Hysteroscopy | 2/469 | 0.4% (95%CI:0.07–1.70) |
Observation → Follow-up with imaging | 1/469 | 0.2% (95%CI:0.01–1.37) |
Biopsy → Medical treatment | 1/469 | 0.2% (95%CI:0.01–1.37) |
DISCUSSION
Gynaecologic MR imaging plays an important role in the care of patients with gynaecologic tumours, often contributing to initial diagnosis, determination of disease extent, treatment selection and treatment follow-up [23]. For example, in patients with endometrial cancer, MR imaging is a valuable tool for pretreatment risk stratification, aiding the identification of patients who stand to benefit from pelvic and para-aortic lymph node dissection [22; 24; 25]. In women with cervical cancer, MR imaging improves the accuracy of the FIGO clinical stage determination, leading to more precise treatment selection and planning [26–28]. MR imaging is crucial for confirming eligibility for fertility-sparing surgical or medical procedures in patients with early-stage endometrial or cervical cancer who desire fertility preservation [29–32], and in women with adnexal masses of an indeterminate nature on ultrasound, MR imaging is the diagnostic problem-solving modality of choice [33; 34].
In our study of 469 MRI examinations, initial reports and second-opinion interpretations by GynOnc MRI subspecialists disagreed in more than a third (181/469) of cases. Some of these discrepancies were not clinically relevant and would not have affected patient care. However, according to the opinions of experienced gynaecologic oncologic surgeons, second-opinion review would have changed some aspect of clinical management in at least one fifth (20–21.5%) of patients, led to a change in the treatment approach for 12.8–15.1% of patients, prevented unnecessary surgery in 6.6–7.5% of patients, improved the surgical approach in 2.5–4.1% of patients, and replaced a non-operative management strategy with a more appropriate, surgical intervention for 1.9–2.3% of patients. The second-opinion report proved correct in the majority (103/124, 83%) of cases for which at least one surgeon found clinically important differences between initial and second-opinion MRI reports. We hypothesize that the focused, subspecialty expertise of the radiologists who performed the interpretations largely accounts for considerable differences in interpretations. This proficiency has likely developed as a result of repeated exposure to a large volume of cases with similar clinical findings and questions, a concept that is supported by prior literature, including GYN imaging. As a part of UK nationwide audit, Duncan et al evaluated the performance of various centres across the UK for the assessment of endometrial cancer stage using MRI. That study found that the centres with higher case load were significantly more accurate at the evaluation of the depth of myometrial invasion using histopathology as a gold standard [35].
To our knowledge, no prior studies have explored the added value of second-opinion radiology subspecialty review of GynOnc MRI examinations. Two previous reports examined the impact of a gynaecological oncology tumour board on the management of women with known or suspected gynaecologic malignancies referred to a tertiary care centre [19; 20]. Cohen et al reported on 509 patients discussed at the meetings of a gynaecological oncology tumour board and found that major discrepancies were seen for 30 of the patients (5.9%) [19]. The study did not specify the total number of patients who had outside imaging or pathology examinations reviewed or what types of examinations were reviewed. Greer et al reported on 215 patients whose cases were presented for radiological review (mostly of CT and PET scans) at a weekly multidisciplinary gynaecologic oncology tumour board; they found that the secondary radiological review led to a new diagnosis of gynaecologic malignancy or upstaging of known cancer in 19 (10%) of patients [20].
Our findings assessing the value of second-opinion image interpretations are in agreement with those of several studies involving other radiology sub-specialties. Gollub et al. reported on 143 body CT scans reinterpreted at a tertiary care cancer centre and found major disagreements between initial and second-opinion interpretations of 24/143 (17%) scans; the disagreements led to changes in management in 5/143 (3.5%) patients [10]. Eakins et al. analyzed 733 paediatric cases for which diagnostic imaging examinations were re-reviewed at a paediatric hospital; they found major discrepancies between initial and second-opinion reports for 168 (21.7%) examinations [8]. The second-opinion interpretations were more accurate than the initial reports in 83 (90%) of 92 discrepant cases with proven diagnoses. In a review of 4,534 neuroradiology cases, Zan et al. found clinically significant discrepancies between initial and second-opinion reports for 347 (7.7%) imaging studies [13]. The second-opinion review was correct in 163 (84%) of 194 discrepant cases with pathologically proven final diagnoses. When comparing initial and second-opinion reports for 396 patients who were referred to surgical oncologists at four academic centres, Dudley et al. found that reports disagreed in 162 (41%) of all cases, and the second-opinion interpretations were correct in 153 (94%) discrepant cases [18]. Brook et al reported on 383 consecutive patients presented at a Radiology Conference of the Division of Oncology and found these second-opinion consultations lead to major changes in management of 37% cancer patients and provided additional important information in up to 50% of patients [14].
Our study had several limitations. The study design is retrospective potentially leading to a number of selection biases; for example, only GYN MRI examinations submitted for second-opinion review were included; it is possible that the rate of clinically relevant discrepancies would have been different if all outside GYN MRI studies were submitted for review. Second, the distinction between clinically important and unimportant discrepancies is subjective. We tried to limit this concern by recruiting two senior gynaecological oncologic surgeons to evaluate all de-identified MRI reports with any discrepancies between initial readings and second-opinion interpretations.
In conclusion, our results indicate that second-opinion review of GynOnc MRI by sub-specialized radiologists can impact patient care, allowing for more informed medical decision-making. In conjunction with previous studies in other imaging disciplines, our findings support the notion that subspecialty training and focused expertise influence patient management in a setting of multidisciplinary, disease-specific team-based care. Furthermore, second-opinion consultations should be viewed as a valuable and reimbursable clinical service.
Key Points.
Outside gynaecologic oncologic MRI examinations are often submitted for the second-opinion review.
One-fifth of MRIs had important discrepancies between initial and second-opinion interpretations.
Second-opinion review of gynaecologic oncologic MRI is a valuable clinical service.
Acknowledgments
The scientific guarantor of this publication is Evis Sala, MD, Ph.D., FRCR. The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article. This research was funded in part through the MSK Cancer Center Support Grant/Core Grant (P30 CA008748). The authors thank Chaya Moskowitz PhD, who kindly provided statistical advice for this manuscript and Ada Muellner MS who provided editorial support. Approval from the Institutional Review Board of Memorial Sloan-Kettering Cancer Center was obtained. Written informed consent was waived by the Institutional Review Board. Methodology: retrospective, performed at one institution.
This study was a part of Melvin D’Anastasi’s doctoral thesis. Yulia Lakhman MD, Melvin D’Anastasi MD, Hedvig Hricak MD PhD, and Evis Sala MD PhD FRCR contributed equally to this study.
Abbreviations and acronyms
- CA
carcinoma
- CI
confidence interval
- CT
computed tomography
- FIGO
International Federation of Gynaecology and Obstetrics
- GYN
gynaecologic
- GynOnc
gynaecologic oncologic
- MR
magnetic resonance
- MRI
magnetic resonance imaging
- PET
positron emission tomography
- SCC
squamous cell carcinoma
- US
ultrasound
References
- 1.Barnes AJ, Unruh L, Chukmaitov A, van Ginneken E. Accountable care organizations in the USA: types, developments and challenges. Health Policy. 2014;118:1–7. doi: 10.1016/j.healthpol.2014.07.019. [DOI] [PubMed] [Google Scholar]
- 2.Dannapfel P, Poksinska B, Thomas K. Dissemination strategy for Lean thinking in health care. Int J Health Care Qual Assur. 2014;27:391–404. doi: 10.1108/IJHCQA-01-2013-0001. [DOI] [PubMed] [Google Scholar]
- 3.Gordon JE, Leiman JM, Deland EL, Pardes H. Delivering value: provider efforts to improve the quality and reduce the cost of health care. Annu Rev Med. 2014;65:447–458. doi: 10.1146/annurev-med-100312-135931. [DOI] [PubMed] [Google Scholar]
- 4.Kinsman L, Rotter T, Stevenson K, et al. “The largest Lean transformation in the world”: the implementation and evaluation of lean in Saskatchewan healthcare. Healthc Q. 2014;17:29–32. doi: 10.12927/hcq.2014.23880. [DOI] [PubMed] [Google Scholar]
- 5.Downs CG, Fowler L, Kolodziej M, et al. The Affordable Care Act: where are we now? An NCCN roundtable. J Natl Compr Canc Netw. 2014;12:745–747. doi: 10.6004/jnccn.2014.0182. [DOI] [PubMed] [Google Scholar]
- 6.Porter ME. What is value in health care? N Engl J Med. 2010;363:2477–2481. doi: 10.1056/NEJMp1011024. [DOI] [PubMed] [Google Scholar]
- 7.Briggs GM, Flynn PA, Worthington M, Rennie I, McKinstry CS. The role of specialist neuroradiology second opinion reporting: is there added value? Clin Radiol. 2008;63:791–795. doi: 10.1016/j.crad.2007.12.002. [DOI] [PubMed] [Google Scholar]
- 8.Eakins C, Ellis WD, Pruthi S, et al. Second opinion interpretations by specialty radiologists at a pediatric hospital: rate of disagreement and clinical implications. AJR Am J Roentgenol. 2012;199:916–920. doi: 10.2214/AJR.11.7662. [DOI] [PubMed] [Google Scholar]
- 9.Erly WK, Ashdown BC, Lucio RW, 2nd, Carmody RF, Seeger JF, Alcala JN. Evaluation of emergency CT scans of the head: is there a community standard? AJR Am J Roentgenol. 2003;180:1727–1730. doi: 10.2214/ajr.180.6.1801727. [DOI] [PubMed] [Google Scholar]
- 10.Gollub MJ, Panicek DM, Bach AM, Penalver A, Castellino RA. Clinical importance of reinterpretation of body CT scans obtained elsewhere in patients referred for care at a tertiary cancer center. Radiology. 1999;210:109–112. doi: 10.1148/radiology.210.1.r99ja47109. [DOI] [PubMed] [Google Scholar]
- 11.Loevner LA, Sonners AI, Schulman BJ, et al. Reinterpretation of cross-sectional images in patients with head and neck cancer in the setting of a multidisciplinary cancer center. AJNR Am J Neuroradiol. 2002;23:1622–1626. [PMC free article] [PubMed] [Google Scholar]
- 12.Loughrey GJ, Carrington BM, Anderson H, Dobson MJ, Lo Ying Ping F. The value of specialist oncological radiology review of cross-sectional imaging. Clin Radiol. 1999;54:149–154. doi: 10.1016/s0009-9260(99)91003-6. discussion 154–145. [DOI] [PubMed] [Google Scholar]
- 13.Zan E, Yousem DM, Carone M, Lewin JS. Second-opinion consultations in neuroradiology. Radiology. 2010;255:135–141. doi: 10.1148/radiol.09090831. [DOI] [PubMed] [Google Scholar]
- 14.Brook OR, Hakmon T, Brook A, Dudnik E, Kuten A, Engel A. The effect of a Radiology Conference consultation on cancer patients management. Ann Oncol. 2011;22:1204–1208. doi: 10.1093/annonc/mdq581. [DOI] [PubMed] [Google Scholar]
- 15.Lu MT, Tellis WM, Avrin DE. Providing formal reports for outside imaging and the rate of repeat imaging. AJR Am J Roentgenol. 2014;203:107–110. doi: 10.2214/AJR.13.10617. [DOI] [PubMed] [Google Scholar]
- 16.Bell ME, Patel MD. The degree of abdominal imaging (AI) subspecialization of the reviewing radiologist significantly impacts the number of clinically relevant and incidental discrepancies identified during peer review of emergency after-hours body CT studies. Abdom Imaging. 2014;39:1114–1118. doi: 10.1007/s00261-014-0139-4. [DOI] [PubMed] [Google Scholar]
- 17.Jordan MJ, Lightfoote JB, Jordan JE. Quality outcomes of reinterpretation of brain CT imaging studies by subspecialty experts in neuroradiology. J Natl Med Assoc. 2006;98:1326–1328. [PMC free article] [PubMed] [Google Scholar]
- 18.Dudley RA, Hricak H, Scheidler J, et al. Shared patient analysis: a method to assess the clinical benefits of patient referrals. Med Care. 2001;39:1182–1187. doi: 10.1097/00005650-200111000-00005. [DOI] [PubMed] [Google Scholar]
- 19.Cohen P, Tan AL, Penman A. The multidisciplinary tumor conference in gynecologic oncology--does it alter management? Int J Gynecol Cancer. 2009;19:1470–1472. doi: 10.1111/IGC.0b013e3181bf82df. [DOI] [PubMed] [Google Scholar]
- 20.Greer HO, Frederick PJ, Falls NM, et al. Impact of a weekly multidisciplinary tumor board conference on the management of women with gynecologic malignancies. Int J Gynecol Cancer. 2010;20:1321–1325. doi: 10.1111/IGC.0b013e3181f5871e. [DOI] [PubMed] [Google Scholar]
- 21.Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998;17:857–872. doi: 10.1002/(sici)1097-0258(19980430)17:8<857::aid-sim777>3.0.co;2-e. [DOI] [PubMed] [Google Scholar]
- 22.Beddy P, Moyle P, Kataoka M, et al. Evaluation of depth of myometrial invasion and overall staging in endometrial cancer: comparison of diffusion-weighted and dynamic contrast-enhanced MR imaging. Radiology. 2012;262:530–537. doi: 10.1148/radiol.11110984. [DOI] [PubMed] [Google Scholar]
- 23.Sala E, Rockall AG, Freeman SJ, Mitchell DG, Reinhold C. The added role of MR imaging in treatment stratification of patients with gynecologic malignancies: what the radiologist needs to know. Radiology. 2013;266:717–740. doi: 10.1148/radiol.12120315. [DOI] [PubMed] [Google Scholar]
- 24.Manfredi R, Mirk P, Maresca G, et al. Local-regional staging of endometrial carcinoma: role of MR imaging in surgical planning. Radiology. 2004;231:372–378. doi: 10.1148/radiol.2312021184. [DOI] [PubMed] [Google Scholar]
- 25.Rechichi G, Galimberti S, Oriani M, Perego P, Valsecchi MG, Sironi S. ADC maps in the prediction of pelvic lymph nodal metastatic regions in endometrial cancer. Eur Radiol. 2013;23:65–74. doi: 10.1007/s00330-012-2575-2. [DOI] [PubMed] [Google Scholar]
- 26.Nicolet V, Carignan L, Bourdon F, Prosmanne O. MR imaging of cervical carcinoma: a practical staging approach. Radiographics. 2000;20:1539–1549. doi: 10.1148/radiographics.20.6.g00nv111539. [DOI] [PubMed] [Google Scholar]
- 27.Okamoto Y, Tanaka YO, Nishida M, Tsunoda H, Yoshikawa H, Itai Y. MR imaging of the uterine cervix: imaging-pathologic correlation. Radiographics. 2003;23:425–445. doi: 10.1148/rg.232025065. quiz 534–425. [DOI] [PubMed] [Google Scholar]
- 28.Sahdev A, Sohaib SA, Wenaden AE, Shepherd JH, Reznek RH. The performance of magnetic resonance imaging in early cervical carcinoma: a long-term experience. Int J Gynecol Cancer. 2007;17:629–636. doi: 10.1111/j.1525-1438.2007.00829.x. [DOI] [PubMed] [Google Scholar]
- 29.Lakhman Y, Akin O, Park KJ, et al. Stage IB1 cervical cancer: role of preoperative MR imaging in selection of patients for fertility-sparing radical trachelectomy. Radiology. 2013;269:149–158. doi: 10.1148/radiol.13121746. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Peppercorn PD, Jeyarajah AR, Woolas R, et al. Role of MR imaging in the selection of patients with early cervical carcinoma for fertility-preserving surgery: initial experience. Radiology. 1999;212:395–399. doi: 10.1148/radiology.212.2.r99au01395. [DOI] [PubMed] [Google Scholar]
- 31.Sahdev A, Jones J, Shepherd JH, Reznek RH. MR imaging appearances of the female pelvis after trachelectomy. Radiographics. 2005;25:41–52. doi: 10.1148/rg.251045047. [DOI] [PubMed] [Google Scholar]
- 32.Wu LM, Xu JR, Gu HY, Hua J, Haacke EM, Hu J. Predictive value of T2-weighted imaging and contrast-enhanced MR imaging in assessing myometrial invasion in endometrial cancer: a pooled analysis of prospective studies. Eur Radiol. 2013;23:435–449. doi: 10.1007/s00330-012-2609-9. [DOI] [PubMed] [Google Scholar]
- 33.Hricak H, Chen M, Coakley FV, et al. Complex adnexal masses: detection and characterization with MR imaging--multivariate analysis. Radiology. 2000;214:39–46. doi: 10.1148/radiology.214.1.r00ja3939. [DOI] [PubMed] [Google Scholar]
- 34.Kinkel K, Lu Y, Mehdizade A, Pelte MF, Hricak H. Indeterminate ovarian mass at US: incremental value of second imaging test for characterization--meta-analysis and Bayesian analysis. Radiology. 2005;236:85–94. doi: 10.1148/radiol.2361041618. [DOI] [PubMed] [Google Scholar]
- 35.Duncan KA, Drinkwater KJ, Frost C, Remedios D, Barter S. Staging cancer of the uterus: a national audit of MRI accuracy. Clin Radiol. 2012;67:523–530. doi: 10.1016/j.crad.2011.10.019. [DOI] [PubMed] [Google Scholar]