Interobserver Agreement for Detection of Malignant Features of Intraductal Papillary Mucinous Neoplasms of the Pancreas on MDCT

Abstract

OBJECTIVE

The purpose of this retrospective study was to measure interobserver agreement in the assessment of malignant imaging features of intraductal papillary mucinous neoplasms (IPMNs) on MDCT.

MATERIALS AND METHODS

Pancreatic protocol CT studies were reviewed for 84 patients with resected IPMNs. Maximal diameter of the dominant cyst, presence of a mural nodule, presence of a solid component, and diameters of the main pancreatic duct (MPD) and common bile duct (CBD) were measured by four radiologists independently. In each patient, the IPMN was classified into one of three types: main duct, branch duct, or mixed IPMN. Interobserver agreement of lesion features was examined using the intraclass correlation coefficient (ICC) for continuous features and Fleiss kappa for categorical features.

RESULTS

The final dataset included 55 branch duct IPMNs, nine main duct IPMNs, and 20 mixed IPMNs. Moderate agreement (κ = 0.458; 95% CI, 0.345–0.564) was observed in assigning branch duct, main duct, or mixed IPMN subtypes. Measurement agreement was substantial to excellent for dominant cyst (ICC = 0.852; 95% CI, 0.777–0.907), MPD (0.753, 0.655–0.837), and CBD (0.608, 0.463–0.724) but only fair to moderate for the detection of the presence of mural nodule (κ = 0.284, 0.125–0.432) or solid component (κ = 0.405, 0211–0.577).

CONCLUSION

Substantial to excellent interobserver agreement in the measurement of cyst diameter, MPD, and CBD support their use for characterizing malignant features of IPMN on MDCT. However, the subjective interpretation of the presence of solid components and mural nodules by individual radiologists was more variable.

Intraductal papillary mucinous neoplasms (IPMNs) of the pancreas are mucin-producing neoplasms arising from the main pancreatic duct or its branches. These lesions exhibit a variable degree of dysplasia, from low-grade to high-grade, or even invasive carcinoma [1]. IPMNs and other cystic pancreatic lesions are increasingly found incidentally during abdominal imaging [2, 3], and their management depends on the suspicion of high-grade dysplasia or invasive carcinoma. Treatment recommendations for patients with IPMNs are based on clinical, radiologic, and histopathologic data. According to international consensus guidelines published in 2006, main duct IPMNs should generally be considered for resection, whereas branch duct IPMNs should be considered for surgery in the presence of symptoms when larger than 3 cm or when harboring mural nodules or a solid component [4]. A 2012 revision to these guidelines suggests that identification of one of several “worrisome features” should prompt a possible endoscopic evaluation, whereas patients with “high-risk stigmata” should be considered for surgical resection. Worrisome features include main pancreatic duct (MPD) size of 5–9 mm, a cyst size greater than 3 cm, a cyst with a non-enhancing mural nodule, abrupt change in duct caliber, and thickened enhancing cyst wall. High-risk stigmata include an MPD of 10 mm or greater, a cyst with an enhancing solid component, or a pancreatic head cyst with obstructive jaundice [5].

Multiple studies form the basis of these recommendations, and imaging features associated with malignancy in IPMNs have been widely reported, including MPD dilatation, size of the cystic lesion, and presence of mural nodules or solid components [6–15]. In practice, application of the consensus guidelines for IPMN relies on an individual radiologist’s ability to assess malignant features of IPMN, which rests on the hypothesis that interobserver agreement is excellent for diagnostic radiologists of varying experience. However, to our knowledge, the extent of interobserver variability among radiologists in detection of these malignant features on MDCT has not been previously studied. High interobserver variability due to the subjective nature of image interpretation may limit the utility of an imaging feature, a modality, or even a set of guidelines [16]. Even the measurement of continuous variables can yield high variability [17] and could be a concern for MPD measurements. One source of potential interpretation variability is the frequent lack of precise definitions of mural nodules and solid components in published guidelines and literature. Thus, the purpose of this retrospective study was to measure interobserver agreement among radiologists in the assessment of malignant imaging features of IPMN on MDCT.

Materials and Methods

Patients

Institutional review board approval was obtained for this HIPAA-compliant retrospective study. Informed consent requirements were waived. The prospectively maintained institutional pancreatic cyst registry was queried for patients evaluated at Memorial Sloan Kettering Cancer Center for a cystic neoplasm of the pancreas from February 1995 to August 2010. Patients were selected who met the following inclusion criteria: pathologic confirmation of IPMN, preoperative MDCT imaging within 6 months of surgery, and CT with multiphasic pancreatic protocol. The database search initially yielded 211 patients who underwent presurgical CT, 120 of whom were excluded for lacking a multiphasic pancreatic protocol CT for review. Of the remaining 91 patients, seven more were excluded, two because imaging was more than 6 months before surgery, one for incomplete coverage of the pancreas on parenchymal phase imaging, two for unresectable disease precluding pathologic evaluation, and two for harboring pancreatic adenocarcinoma separate from a branch duct IPMN that remained in situ. The remaining 84 patients (43 men and 41 women; median age, 70 years; range, 30–86 years) with pathologic confirmation of IPMN had undergone multiphasic pancreatic protocol CT within 6 months before surgery (average time between CT and resection, 26.4 days; range 2–97 days) and were included in the final cohort (Table 1).

TABLE 1.

Intraductal Papillary Mucinous Neoplasm (IPMN) Subtype and Pathologic Classification

IPMN Subtype	Low-Grade Dysplasia	Intermediate-Grade Dysplasia	High-Grade Dysplasia	Invasive Cancer
Branch duct	8 (14.5)	26 (47.3)	10 (18.2)	11 (20.0)
Main duct	0	1 (11.1)	4 (44.4)	4 (44.4)
Mixed	0	2 (10.0)	9 (45.0)	9 (45.0)
Total	8 (9.5)	29 (34.5)	23 (27.4)	24 (28.6)

Note—Data are number with percentage in parentheses.

Pathologic Analysis

All resected IPMNs were reviewed by a gastrointestinal pathologist and graded according to the World Health Organization classification on the basis of the most severely dysplastic region as tumors with low-grade, intermediate-grade, or high-grade dysplasia or as harboring invasive carcinoma.

Imaging Technique

To qualify as a multiphasic pancreatic protocol CT, an examination required inclusion of thin reconstruction images (≤ 2.5-mm slice thickness) obtained during the pancreatic parenchymal phase. All studies were performed at our institution, on MDCT scanners (LightSpeed 16 or VCT 64, GE Healthcare) and all images were available for review on a PACS (Centricity PACS 2.0, GE Healthcare). Most studies were performed using our department’s standard MDCT pancreatic protocol after 2005: 65 studies included multiplanar images, whereas 19 examinations (before 2005) only included images in one plane for review. In our standard pancreatic protocol after 2005, patients were imaged before and after IV administration of 150 mL of iodinated contrast material (iohexol, Omnipaque 300, GE Healthcare) injected at 4 mL/s, with water as oral contrast material. Axial unenhanced 5-mm slices, parenchymal phase 2.5-mm slices through the pancreas, and portal venous phase 2.5-mm slices through the entire abdomen were obtained. In addition, 2.5-mm coronal and sagittal reconstructed images from the parenchymal and portal venous phases were generated. The following scanning parameters were used: table speed, 0.984–1.375 (pitch, 39.37–27.50 mm); rotation time, 0.7–0.8 ms; and scanning delays of 40 seconds for the parenchymal phase and 80 seconds for the portal venous phase. The same scanning parameters were used for the 19 examinations with the axial plane only available for review. However, the slice thickness was more variable for parenchymal phases (1.25–2.5 mm) and portal venous phase (2.5–5.0 mm).

Imaging Interpretation

Radiologists interpreting the images were aware of the clinical diagnosis of IPMN but were blinded to pathologic and any additional clinical data. Four radiologists of differing experience levels independently reviewed the images of all patients. The images were sorted in a randomized order unique to each reviewer. The radiologists included a fellow in training for body imaging; two junior faculty radiologists (junior faculty radiologists 1 and 2) with 3 and 5 years of experience after an abdominal imaging fellowship, respectively; and a senior faculty radiologist with 20 years of experience.

The following imaging features were assessed with both the pancreatic parenchymal and portal venous phases: maximal diameter of the dominant cyst, presence and size of a mural nodule, presence and size of a solid component, diameter of the MPD, and caliber (diameter) of the common bile duct (CBD). The presence or absence of biliary stents was also recorded. Measurements were recorded on whichever plane yielded the largest long-axis diameter for lesions and largest caliber for ducts. A solid component was defined as an area of abnormal attenuation on either the pancreatic parenchymal or portal venous phase adjacent to the cystic lesion. A mural nodule was defined as a wall-based rounded soft-tissue density or enhancing lesion projecting into the cystic lesion not thought to represent a confluence of septa. All dimensions (for dominant cyst, mural nodule, solid component, pancreatic duct, and CBD) were recorded in millimeters. The presence or absence of a mural nodule or solid component was recorded on a 5-point scale: 1 = definitely absent, 2 = probably absent, 3 = possibly present, 4 = probably present, and 5 = definitely present. For ease of analysis and interpretation, the presence or absence of these features was then dichotomized as absent (score 1–3) or present (score 4 or 5).

For each patient, each radiologist classified the IPMN into one of three types: main duct, branch duct, or mixed IPMN. Each reader was instructed to make this assessment on the basis of overall morphology and MPD size, without using a predetermined threshold diameter for the pancreatic duct. When a reader designated a main duct IPMN, no separate measurable cyst was identified and recorded; in cases of suspected mixed IPMN, the largest discrete cyst was measured. An IPMN type interpretation was deemed a “consensus agreement” if at least three readers identified the same type; in cases of initial lack of consensus, the three faculty radiologists reviewed the images again in a subsequent session and after further discussion reached a consensus.

On the basis of the pathologic results, we calculated the proportions of IPMNs harboring invasive cancers for each IPMN subtype by consensus. We compared all IPMNs with invasive cancers to those without with respect to each imaging feature assessed by all four readers. If the agreement was sufficiently high, we averaged the feature studied across all four readers before comparing invasive to noninvasive IPMNs. Otherwise, we compared the feature studied individually for each reader. We then repeated this analysis for the subset of branch duct IPMNs.

Statistical Analysis

We tabulated patient characteristics including age, sex, and pathology. The Fisher exact test was used to examine associations between categoric variables, and the Kruskal-Wallis test was used to examine associations between a categoric and continuous variable.

Interobserver agreement of lesion features was examined using the intraclass correlation coefficient (ICC) for continuous features and unweighted Fleiss kappa for categoric features. Continuous features included diameter of the dominant cyst, MPD, and CBD, measured in millimeters, whereas categoric features included presence of mural nodule and solid component. Median values are presented for the dominant cyst, MPD, and CBD because these measures are not normally distributed. Nonparametric methods or log-transformation was used for all analyses of these measures. Because mural nodule and solid component diameters were measured only if a score of 4 or 5 was indicated on the presence-absence scale for each feature, these two continuous variables were not used in any analyses due to the subsequently small number of observations measured by all readers.

To compute the ICC, a random-effects model was fit with a subject effect and a rater effect. The outcome was the log-transformed measurement. The resulting variance components were converted back to the original scale, and the ICC was calculated, and 95% CIs for the ICC and kappa values were calculated using percentile intervals from 1000 bootstrap samples.

Agreement statistics are only available for features with no missing values for all four readers. Agreement values are interpreted as follows: < 0, poor agreement; 0.01–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.81–1.00, excellent agreement [18]. Interobserver agreements were also recalculated for the 65 cases that had available multiplanar reformatted imaging.

All analyses were repeated for the subgroup of patients with branch duct IPMNs determined by consensus among the four readers. All analyses were conducted separately for each reader. Statistical significance was defined as p < 0.05. Statistical analyses were conducted using SAS, version 9.2 (SAS Institute) and R, version 2.11.0 (R Development Core Team), including the “irr,” “lme4,” and “boot” packages.

Results

From the radiologists’ consensus interpretations, the dataset contained 55 branch duct IPMNs (22 men and 33 women; median age, 71 years; range, 47–86 years), nine main duct IPMNs (seven men and two women; median age, 74 years; range, 30–86 years), and 20 mixed IPMNs (14 men and six women; median age, 67.5 years; range, 50–83 years). The distribution of IPMN types with respect to each pathologic diagnosis is summarized in Table 1. A statistically significantly higher proportion of invasive cancers was found among main duct IPMNs and mixed IPMNs (45%, 13 of 29) than branch duct IPMNs (20%, 11 of 55) (p = 0.023). In addition, there was a statistically higher proportion of IPMNs with either invasive cancer or high-grade dysplasia in main duct IPMNs and mixed IPMNs (90%, 26 of 29) compared with branch duct IPMNs (38%, 21 of 55) (p < 0.001).

Interobserver agreement results in the assessment of malignant imaging features of IPMN are summarized in Tables 2 and 3. Moderate agreement was observed in assigning branch duct, main duct, or mixed IPMN subtypes. There was excellent agreement for the measurement of cystic component diameter and substantial agreement for measurement of the MPD and CBD diameters. There was fair agreement in the detection of the presence of mural nodules (Fig. 1), moderate agreement in the detection of the presence of solid components, and moderate agreement for detection of either mural nodules or solid components as a single category. Interobserver agreements were similar when the analysis was limited to the three faculty readers (excluding the fellow in training) or when limiting the analysis to the 65 cases with available multiplanar reformatted images (Tables 2 and 3).

TABLE 2.

Size Features of 84 Intraductal Papillary Mucinous Neoplasms (IPMNs) by Reader

Feature	Reader				Agreement for All Readers (95% CI)a	Agreement for Three Faculty Readers (95% CI)	Agreement for All Readers for 65 CT Studies With Multiplanar Images
	Fellow	Junior Faculty 1	Junior Faculty 2	Senior Faculty
Maximal dimension (mm)	35 (14–83)	35 (13–88)	35 (10–84)	32 (11–83)	0.85 (0.78–0.91)	0.85 (0.72–0.89)	0.85 (0.70–0.90)
Main pancreatic duct size (mm)	5 (2–25)	6 (1–28)	5 (1–22)	4 (1–29)	0.75 (0.66–0.84)	0.73 (0.62–0.84)	0.74 (0.60–0.85)
Common bile duct size (mm)	6 (3–23)	7 (1–24)	6 (1–22)	6 (2–25)	0.61 (0.46–0.72)	0.62 (0.38–0.75)	0.71 (0.36–0.77)

Note—Except where otherwise indicated, data are median with range in parentheses.

^aAgreement measured by intraclass correlation coefficient for continuous data and by Fleiss kappa for categoric data.

TABLE 3.

Malignant Features of 84 Intraductal Papillary Mucinous Neoplasms (IPMNs) by Reader

Feature	Reader				Agreement for All Readers (95% CI)a	Agreement for Three Faculty Readers (95% CI)	Agreement All Readers for 65 CT Studies With Multiplanar Images
	Fellow	Junior Faculty 1	Junior Faculty 2	Senior Faculty
IPMN subtype					0.46 (0.35–0.56)	0.51 (0.39–0.62)	0.46 (0.32–0.58)
Branch duct	48 (57.1)	55 (65.5)	53 (63.1)	69 (82.1)
Main duct	14 (16.7)	9 (10.7)	9 (10.7)	10 (11.9)
Mixed	22 (26.2)	20 (23.8)	22 (26.2)	5 (6.0)
Nodule					0.28 (0.13–0.43)	0.31 (0.12–0.48)	0.27 (0.11–0.46)
No	74 (88.1)	62 (73.8)	72 (85.7)	62 (73.8)
Yes	10 (11.9)	22 (26.2)	12 (14.3)	22 (26.2)
Solid component					0.41 (0.21–0.58)	0.39 (0.19–0.57)	0.40 (0.19–0.59)
No	73 (86.9)	74 (88.1)	69 (83.1)	65 (77.4)
Yes	11 (13.1)	10 (11.9)	14 (16.9)	19 (22.6)
Nodule or solid component					0.45 (0.30–0.58)	0.47 (0.33–0.60)	0.45 (0.30–0.61)
No	66 (78.6)	54 (64.3)	60 (71.4)	46 (54.8)
Yes	18 (21.4)	30 (35.7)	24 (28.6)	38 (45.2)

Note—Except where otherwise indicated, data are number with percentage in parentheses.

^aAgreement measured by intraclass correlation coefficient for continuous data and by Fleiss kappa for categoric data.

Fig. 1.

Interobserver agreement for mural nodules and solid components. These cases show challenge in determining presence of mural nodules and solid components in intraductal papillary mucinous neoplasms (IPMNs) and potential for interobserver variability.

A and B, Contrast-enhanced parenchymal phase CT images in coronal plane in two different patients with IPMNs. Mural nodules (arrow) were interpreted as present by two of four readers in A and by four of four readers in B.

C and D, Contrast-enhanced parenchymal phase CT images in coronal plane in two different patients with IPMNs. Solid components (arrow) were interpreted as present by two of four readers in C and by four of four readers in D.

A similar set of interobserver agreement analyses performed on the subgroup of 55 patients with branch duct IPMNs (33 women and 22 men; mean age, 71 years; range, 47–86 years) (Tables 4 and 5) again revealed excellent agreement for diameter measurement of the cystic component; substantial agreement for measurements of the MPD and CBD; and fair agreement for the assessment of mural nodules, solid components, and either mural nodules or solid components.

TABLE 4.

Size Features of 55 Branch Duct Intraductal Papillary Mucinous Neoplasms (IPMNs) by Reader

Feature	Fellow	Junior Faculty 1	Junior Faculty 2	Senior Faculty	Agreement (95% CI)a
Maximal dimension (mm)	34 (15–83)	33 (13–88)	31 (13–84)	31 (11–83)	0.91 (0.84–0.94)
Main pancreatic duct size (mm)	3 (2–18)	4 (1–17)	3 (1–9)	3 (1–16)	0.60 (0.43–0.72)
Common bile duct size (mm)	6 (3–21)	7 (3–24)	6 (2–22)	5 (2–25)	0.65 (0.49–0.76)

Note—Except where otherwise indicated, data are median in parentheses.

^aAgreement measured by intraclass correlation coefficient for continuous data and by Fleiss kappa for categoric data.

TABLE 5.

Malignant Features of 55 Branch Duct Intraductal Papillary Mucinous Neoplasms (IPMNs) by Reader

Feature	Fellow	Junior Faculty 1	Junior Faculty 2	Senior Faculty	Agreement (95% CI)a
Nodule					0.22 (0.001–0.43)
No	53 (96.4)	42 (76.4)	52 (94.5)	41 (74.5)
Yes	2 (3.6)	13 (23.6)	3 (5.5)	14 (25.5)
Solid component					0.38 (0.03–0.67)
No	52 (94.5)	53 (96.4)	48 (87.3)	50 (90.9)
Yes	3 (5.5)	2 (3.6)	7 (12.7)	5 (9.1)
Nodule or solid component					0.40 (0.18–0.57)
No	50 (90.9)	40 (72.7)	45 (81.8)	37 (67.3)
Yes	5 (9.1)	15 (27.3)	10 (18.2)	18 (32.7)

Note—Except where otherwise indicated, data are number with percentage in parentheses.

^aAgreement measured by intraclass correlation coefficient for continuous data and by Fleiss kappa for categoric data.

After we established substantial to excellent interobserver agreement in assessment of dominant cyst, MPD, and CBD, we averaged measurements of these features across the four readers for each individual patient and rounded to the nearest integer value. The medians of these reader-averaged feature measures were then compared between IPMN harboring invasive cancer and IPMN harboring only varying degrees of dysplasia (Fig. 2). The median diameter of invasive IPMNs was 43 mm compared with 33 mm for noninvasive IPMNs (p = 0.002). The median MPD diameter was also higher for invasive versus noninvasive IPMN (MPD: 10.5 vs 4 mm, p < 0.001). Excluding a single patient’s CBD measurement because of decompression of the biliary system by a stent, the median CBD diameter was also significantly larger for invasive IPMNs compared with noninvasive IPMNs (CBD: 10 vs 6 mm, p = 0.001).

Fig.2.

Boxplots show maximal cyst diameter (D), main pancreatic duct (MPD), and common bile duct (CBD) values for invasive (I) versus noninvasive (NI) intraductal papillary mucinous neoplasms.

To evaluate the association between MPD diameter and invasive malignancy in a manner suggested by the 2012 consensus guidelines, we grouped lesions into MPD size categories (< 5 mm, 5–9 mm, and ≥ 10 mm) and evaluated each category for the percentage of invasive malignancy. The categories had statistically significant differences in their fraction of invasive malignancies (p < 0.001): For those 36 patients with MPD less than 5 mm, four (11%) patients had invasive cancers, and the remaining 32 (89%) patients did not; for the 26 patients with MPD of 5–9 mm, six (23%) patients had invasive cancers, and 20 (77%) patients did not; and for the 22 patients with MPD of 10 mm or greater, 14 (64%) patients had invasive cancers, and the remaining eight (36%) patients did not.

The same analyses were performed on the subgroup of 55 patients considered to have branch duct IPMNs (11 harboring invasive cancers and 44 with varying degrees of dysplasia only). The median diameter of invasive branch duct IPMNs was 37 mm compared with 31.5 mm for those without invasive disease (p = 0.026). The median MPD caliber was also higher for invasive versus noninvasive branch duct IPMNs (MPD: 6.0 vs 3.0 mm, p = 0.044). Again excluding measurement of the single patient’s stent-decompressed biliary system, the median CBD diameter was significantly larger in invasive IPMNs than in noninvasive IPMNs (CBD: 10 vs 6 mm, p = 0.015). MPD size category correlated with frequency of invasive cancers in the branch duct IPMN subgroup as well. Among the 19 patients in the branch duct IPMN subgroup with MPD of 5 mm or less (of which, only one was ≥ 10 mm), seven (37%) harbored invasive carcinoma and 12 (63%) did not, whereas among the 36 branch duct IPMN patients with an MPD less than 5 mm, only four patients harbored invasive carcinoma (11%) and 32 (89%) did not (p = 0.035).

Because the interobserver agreement for presence or absence of mural nodule or solid component was only fair to moderate, we analyzed the frequency of these features for each reader individually in all invasive and noninvasive IPMNs and the subgroup of all patients with branch duct IPMNs. There was a statistically significantly higher presence of either mural nodule or solid component in invasive versus noninvasive IPMNs for all readers (fellow: 50% vs 10%, p < 0.001; junior faculty radiologist 1: 63% vs 25%, p = 0.002; junior faculty radiologist 2: 54% vs 18%, p = 0.003; and senior faculty radiologist: 92% vs 27%, p < 0.001). In the subgroup of 55 branch duct IPMN patients, a similar result was obtained: Presence of either mural nodule or solid component was detected at a significantly higher rate in invasive than in noninvasive branch duct IPMNs for three of the four readers: fellow, 36% vs 2%, p = 0.004; junior faculty radiologist 2, 55% vs 9%, p = 0.002; and senior faculty radiologist, 81% vs 21%, p < 0.001). For junior faculty radiologist 1, the difference was not statistically significant (46% vs 23%, p = 0.149).

The detection of each of the features of mural nodule and solid component was assessed separately. For mural nodule, no statistically significant association between the presence of this feature and invasive histology was observed for any reader (all, p > 0.05), but there was a significantly higher presence of solid component in invasive versus noninvasive IPMNs for all readers (fellow, 42% vs 2%; junior faculty radiologist 1, 29% vs 5%; junior faculty radiologist 2, 39% vs 8%; and senior faculty radiologist, 54% vs 10% [all readers, p < 0.01]). In the subgroup of 55 branch duct IPMN patients, a similar result was obtained. Presence of mural nodule only differed significantly between invasive and noninvasive branch duct IPMNs for one reader, the fellow (18% vs 9%, p = 0.037), whereas solid component was detected at a statistically significantly higher rate in invasive than in noninvasive branch duct IPMNs for three of the four readers (junior faculty radiologist 1: 18% vs 0%, p = 0.037; junior faculty radiologist 2: 46% vs 5%, p = 0.002; and senior faculty radiologist: 36% vs 2%, p = 0.004). For the fellow, the difference was not statistically significant (18% vs 2%, p = 0.099).

Discussion

Several imaging features of IPMNs have been associated with malignancy; however, to be useful in directing patients to different management options, a feature must not only be significantly associated with malignancy but also reproducibly identifiable. This study is the first to examine the interobserver agreement in the assessment of malignant imaging features of IPMN on multiphasic MDCT. Our data show substantial to excellent interobserver agreement in the assessment of malignant features that are measured as continuous variables, including the sizes of the MPD and CBD as well as the dominant cyst, the diameter of the largest cystic component. Substantial interobserver agreement in measuring MPD size is reassuring because the new consensus guidelines [5] emphasize the classification of risk of IPMNs for invasive cancers by MPD size. In our study the assessment of IPMN subtype as branch duct versus main duct or mixed IPMN had only moderate interobserver agreement. This result further indicates that distinguishing between IPMN subtypes by imaging is subjective and less reliable than the measurement of MPD size and supports the new guidelines in suggesting that management of IPMN should rely more on MPD size measurements [5]; mixed IPMNs are no longer treated as a separate entity.

The assessment of mural nodules and solid components had only fair to moderate interobserver agreement when including all IPMNs in our study and only fair agreement when the analysis was limited to the subgroup of patients with branch duct IPMNs. Despite these levels of interobserver agreement, solid component was associated with malignancy for each observer. It would be worthwhile investigating whether the use of this parameter may be best restricted to a consensus evaluation, such as in a tumor board conference setting, or whether the use of MRCP or endoscopic ultrasound would further help. Mural nodules, on the other hand, were not only detected with fair interobserver agreement but also were not significantly associated with invasive malignancy, raising questions about the overall value of interpreting this feature on MDCT. Because of this result, further sub-classification and analysis of enhancing versus nonenhancing nodules was not undertaken. Also not undertaken was analysis of other features previously associated with aggressive IPMN biology, such as wall irregularity and thickened enhancing septa because the current study focused on parameters included in the 2012 international consensus guidelines [4].

Our results emphasize the importance of measuring interobserver variability in studies assessing the diagnostic performance of radiologic studies, such as MDCT, in detecting malignancy. Ogawa et al. [19], in whose study two readers evaluated each lesion in consensus, recognized the importance of prospective studies to evaluate interobserver agreement. Most published studies do not describe in detail the methods by which malignant features of IPMNs are identified. For instance, mural nodules have been found by multiple studies to be significantly associated with malignancy [6, 9, 11, 19–27], but a review of the methodology in these studies reveals little detail on how mural nodules were defined or identified. Some studies only indicate that a nodule was identified by a combination of cross-sectional imaging and endoscopy without specifying how many and when or how many times endoscopy was performed. More recently, two meta-analyses on IPMNs have been published, the first on branch duct IPMNs [28] and the second on branch and main duct IPMNs [29]. The first meta-analysis deemed mural nodules the feature most associated with malignancy but recognized that enhancement of mural nodules was not addressed in many included studies and conceded that the definition of mural nodules remains a problem in the literature. The second meta-analysis held cyst size to be the feature most associated with malignant IPMNs, found the presence of mural nodules a feature less strongly predictive of malignancy, and also noted that mural nodules were only defined in two of the included studies (as protuberances and papillary projections). In our own study, even with the definition of a mural nodule as a wall-based soft-tissue density projecting into a cystic lesion, the agreement among four radiologists was only fair, highlighting the potential limitation of asking an individual radiologist to evaluate this feature on MDCT.

Although it was not the primary aim of our study, we also examined the rates of invasive cancers in our 84 patients with IPMNs. A higher proportion of invasive cancers was observed in IPMNs with larger MPD size; more than half (64%) in IPMNs with MPD of 10 mm or greater versus 24% in those with MPD measuring 5–9 mm and 11% in IPMNs with MPD less than 5 mm. These results are consistent with prior studies [7, 8, 11, 13, 19–21, 23, 26] and support recent updates to the consensus guidelines [5] distinguishing between IPMNs with high risk stigmata (MPD ≥ 10 mm) or worrisome features (MPD 5–9 mm). We also found a higher average CBD diameter among IPMNs with invasive tumors (≥ 10 mm) than among those without invasive cancers (6 mm), which is consistent with prior studies [8, 9, 11, 26].

Although CBD size is not a component of the consensus guidelines, it may serve as a surrogate for obstructive jaundice, which is considered a high-risk sign. Similarly, among branch duct IPMNs, the small but statistically significant difference in the average cyst size between lesions harboring invasive (37 mm) and noninvasive (31 mm) histology support the guideline assignment of a 3-cm threshold for designation of a worrisome feature. In our study, despite the fair to moderate interobserver agreement in the detection of solid components, there was a higher proportion of invasive IPMNs with solid components detected by all four readers compared with noninvasive IPMNs. The detection of mural nodules on MDCT also suffered from fair interobserver agreement and appears to be of limited use; the presence of mural nodules was more frequent in invasive versus noninvasive branch duct IPMNs when interpreted by only one reader of the four.

Interestingly, for both mural nodules and solid components, there was a trend toward greater assignment of either mural nodules or solid components to all IPMNs with increased seniority of the reader (Tables 2–5). It is possible that with increasing experience, radiologists in our study are consciously or unconsciously hoping to improve their sensitivity for malignancy at the expense of specificity. We did not attempt to perform multivariable analyses to determine imaging predictors of invasive cancers in our study because of our small sample size.

This study was limited by its retrospective nature. A prospective study to validate the interobserver variability in detection of malignant features in IPMN is still warranted. The number of available imaging planes varied between studies because our department protocols for CT of the pancreas changed in 2005 to include routine coronal and sagittal reformats. Due to the retrospective nature of this study, lack of available thin-section source images precluded additional multiplanar or oblique reformats. This issue may have caused underestimation of interobserver agreement, for instance, in the assessment of mural nodules. Nonetheless, 65 of 84 CT studies did include multiplanar reformatting and all were performed with multiple enhancement phases.

Our retrospective analysis was also limited to pathologically confirmed IPMNs, a selection bias likely increasing the proportion of IPMNs with malignant features in our study population relative to a prospectively recruited population of patients with pancreatic cystic lesions followed by imaging. This bias could potentially reduce interobserver agreement in the detection of mural nodules and solid components in the context of a prospective study assessing all patients with pancreatic cystic lesions. In practice, equivocal cases at our institution are reviewed in consensus during a multidisciplinary conference that includes radiologists, gastroenterologists, and surgeons who have experience in the management of IPMNs. Equivocal cases may also undergo further endoscopic evaluation and biopsy before surgical options are considered.

In summary, our data support the robustness of MDCT features that depend on measurements, including cyst diameter and MPD and CBD caliber, supporting their use as outlined in the recent international consensus guidelines on management of IPMNs. Our data also suggest caution in the subjective interpretation of the presence of solid components and especially of mural nodules by individual radiologists. Use of these features may perhaps be best delegated to a multidisciplinary conference in which clinical information and additional studies may impact the overall assessment. A prospective study evaluating the added value of consensus evaluation over an individual radiologist’s assessment of IPMNs on cross-sectional imaging may be worth pursuing.