Inter-observer reproducibility of histologic parameters of melanoma deposits in sentinel lymph nodes: implications for management of patients with melanoma

Abstract

Background

Histologic parameters of melanoma deposits in sentinel lymph nodes (SLNs) have been shown to be predictive of the presence or absence of tumor in non-SLNs and clinical outcome, but assessment of these parameters is prone to inter-observer variation.

Methods

Histologic sections of 44 SLNs containing metastatic melanoma were examined by 7 pathologists. Parameters assessed included cross-sectional area of tumor deposits, cross-sectional area of SLNs, percentage of SLN area involved by tumor calculated from the two previous parameters, estimated percentage of SLN area involved by tumor, tumor penetrative depth (TPD), location of tumor within the SLN, and presence of extracapsular spread (ECS). Levels of inter-observer agreement were measured using intraclass correlation coefficients (ICC).

Results

There was good to excellent inter-observer agreement on measurement of quantitative parameters: maximum size of largest tumor deposits, calculated area of 3 largest tumor deposits, percent area of SLN involved by tumor and TPD (ICC 0.88, 0.73, 0.68 and 0.83, respectively). There was moderate agreement on the evaluation of subcapsular versus non-subcapsular location of tumor deposits (ICC = 0.50). Agreement on assessment of ECS was fair (ICC = 0.39).

Conclusions

Assessment of some of the quantitative parameters was highly reproducible between pathologists. However, evaluation of the location of tumor deposits within SLNs and assessment of ECS was less reproducible. Clearer definitions and training can be expected to improve the reproducibility of assessment. These results have important implications for the reliability and reproducibility of these parameters in staging, prediction of outcome, and clinical management of melanoma patients.

The modern sentinel lymph node (SLN) biopsy procedure was developed in the late 1980s and early 1990s.¹ The SLN biopsy (SLNB) procedure in melanoma patients is a highly accurate staging method and the tumor-harboring status of the SLN is the most important prognostic factor for melanoma patients with early stage disease.^2-12 In clinically nodenegative patients, complete regional lymph node dissection (CLND) is now restricted in most centers to patients with demonstrated metastatic disease in SLNs, sparing the majority of patients major surgery with its associated anesthetic risks and potential morbidity (acute wound problems, nerve injury and chronic lymphedema).¹³

Only a minority (15-30%) of patients with positive SLNs have additional lymph node involvement in subsequent CLND specimens.^14-25 If it could be reliably determined which patients were likely to harbor tumor in non-SLNs, the remaining patients could be safely spared CLND and its potential morbidity. Prior studies have evaluated clinical and pathologic features (features of the primary tumor, number of positive SLNs and histologic characteristics of SLN tumor deposits) in an attempt to predict which SLN-positive patients are likely to have tumor in regional non-SLNs. These studies found that patient age,²⁶ gender,²² site of primary tumor,^{17, 24, 27} primary tumor (Breslow) thickness,^{17, 20, 22, 27, 28} Clark level of invasion,²⁷ ulceration,²⁰ primary tumor mitotic rate,^{18, 22} absence of regression,²⁰ and the number of positive SLNs^{16, 22, 23, 27} were significantly predictive of the presence of metastatic melanoma in non-SLN. Histologic parameters of SLN metastases that have been assessed include the size of metastases, tumor penetrative depth (TPD, also known as maximum subcapsular depth and centripetal thickness), the location of SLN tumor deposits in the SLN, the percentage cross-sectional area of the SLN involved and the presence of extracapsular spread. Many of these parameters have been shown to be predictive of non-SLN status and clinical outcome (Table 1).^{15, 17, 21, 22, 26, 28-30} The power of individual features of melanoma metastases in SLN to predict tumor in non-SLN and survival reported in some studies has not been reproduced in others.^{18, 21, 22, 26, 31}

Table 1.

Studies of predictive parameters of metastatic melanoma in SLN

First author (year)	Number of patients with positive non- SLNs/Total number of patients undergoing SLN biopsy (%)	Characteristics of SLN tumor deposits
		Parameters assessed	Parameters predictive of non-SLN metastasis
			Univariate analysis	Multivariate analysis
Reeves (2003)20	16/98 (16% )	Maximum size of SLN metastasis size, SU score, Location of SLN deposits (subcapsular vs non-subcapsular)	SLN metastasis size, SU score, non- subcapsular location	SU score
Dewar (2004)29	24/146 (16%)	Location of SLN metastasis	Non-subcapsular location	Not performed
Starz (2004)24, 41	24/80 (30%)	Simplified S classification (which is a grouped classification of TPD into ≤0.30mm , 0.31-1.00mm and >1.00mm))	Simplified S classification (TPD)	Simplified S classification (TPD)
Lee (2004)27	46/191 (24%)	SLN metastasis size	SLN metastasis size, especially ≥2mm	SLN metastasis size, especially ≥2mm
Cochran (2004)28	19/90 (21%)	% area of SLN occupied by tumor (assessed using a computer-assisted image analysis program); density of interdigitating dendritic cells/mm2 in the nodal paracortex	% area of SLN (especially ≥4%, but also to a lesser extent, 1-4%) occupied by metastasis, dendritic cell density	% of area of SLN occupied by metastasis
Scolyer (2004)15	24/140 (17%)	Maximum diameter and area of largest SLN metastasis, % area of SLN occupied by tumor (calculated from micrometer measurements), TPD, effacement of SLN architecture by extensive melanoma deposit(s), ECS, presence of melanoma cells in perinodal lymphatics	SLN metastasis area (>10mm2), TPD (>2mm ), effacement of nodal architecture, presence of melanoma cells in perinodal lymphatics	Not performed
Sabel (2005)22	34/221 (15%)	SLN tumor burden (classified as micrometastases or macrometastases), ECS	ECS	ECS
Fink (2005)38	4/26 (15%)	Simplified S classification (which is a grouped classification of TPD into ≤0.30mm, 0.31-1.00mm and >1.00mm))	Simplified S classification (TPD)	Not performed
Vuylsteke (2005)25	19/71 (27%)	% area of SLN occupied by tumor (assessed using an interactive video morphometry system);	SLN metastasis area	SLN metastasis area
Pearlman (2006)17	17/80 (21%)	SLN metastasis size (stratified into 2 groups, ≤2mm or >2mm; if multiple deposits were present, the size of the largest deposit determined the group into which the case was stratified)	SLN metastasis size	Not performed
Govindarajan (2007)21	20/127 (16%)	Size of largest SLN metastasis; Location of SLN metastasis	SLN metastasis size (particularly ≥0.20mm)	SLN metastasis size (particularly ≥0.20mm)
Debarbieux (2007)26	22/98 (22%)	Size of largest SLN metastasis (measured as longest dimension of metastasis and dimension perpendicular to this, known as diameter 2); TPD; Location of SLN metastasis; ECS; presence of tumor in capsular lymphatic vessels	SLN metastasis size (diameter 2), TPD	SLN metastasis size (diameter 2)
Page (2007)18	19/70 (24%)	SLN tumor burden (classified as >2mm, ≤2mm, clusters of cells in subcapsular or parenchymal zones, or isolated melanoma cells in subcapsular sinuses)	None	Not performed
Roka (2008)31	18/85 (21%)	Size of SLN metastases (classified as >2mm and ≤2mm)	None	Not performed
Rossi (2008)39	20/96 (21%)	Size of SLN metastases (classified as >2mm and ≤2mm); TPD, Simplified S classification, location of SLN metastasis	TPD, simplified S classification, SLN metastasis size, non- subcapsular location and extensive SLN involvement	TPD
Satzger (2008)40	28/180 (16%)	Size of SLN metastasis (diameter of largest deposit measured by ocular micrometer and % of SLN area occupied by tumor measured with a computer- morphometry system), TPD, capsular involvement by tumor, ECS	SLN tumor size (largest diameter), TPD, capsular involvement, ECS	% of area of SLN occupied by metastasis, perinodal lymphatic tumor

SLN = sentinel node; % = percentage; SU score = derived from primary tumor ulceration status and size of SLN metastasis >2mm;²⁰ TPD = tumor penetrative depth; ECS = extracapsular spread

Accurate assessment, classification and measurement of the histologic characteristics of SLN tumor deposits requires pathologists to make subjective judgements, and is therefore prone to inter-observer variation. The amount of such variation can be assessed by determining a reliability index, which is an indicator of the level of agreement between observers with adjustment for the degree of agreement that could be expected on the basis of chance. It is usually expressed as an intraclass correlation coefficient (ICC) or kappa score.^32-35 ICC/kappa equals 0 if the observed level of agreement could be expected by chance, and equals 1 if the observers always agree completely.

There have to date been no published studies reporting the inter-observer reproducibility of evaluation of histologic parameters of melanoma deposits in SLNs. Reproducible assessment of these parameters by pathologists is essential to ensure clinical applicability and predictive accuracy, as well as permitting comparison of different studies evaluating these approaches. Standardization is critical if these parameters are to be used to select individual patients who might safely be spared CLND.³⁶ In this study, we attempted to determine the level of inter-observer agreement between pathologists at different institutions in the assessment of a range of histologic characteristics of SLN melanoma deposits. The aim was to assess the level of agreement in the assessment of these parameters, and to identify any areas of difficulty or poor agreement. Identification of such problem areas could potentially lead to clearer definitions and criteria, and/or quality assurance measures such as training slides or microscope consensus sessions, resulting in improved interobserver agreement and standardization in the evaluation of these parameters.

METHODS

Slides from forty-four SLNs that contained metastatic melanoma deposits were retrieved from the records of the Department of Anatomical Pathology, Royal Prince Alfred Hospital, Sydney, Australia. The cases were derived from patients with primary cutaneous melanoma who underwent SLNB at the Sydney Melanoma Unit between January 2001 and December 2004. The cases were chosen at random, in order to represent a range of sizes and morphologies of tumor deposits. Three slides from each SLN (one stained with haematoxylin-eosin, and one each stained immunohistochemically with antibodies to S-100 protein and HMB-45) were chosen for assessment. The slides were de-identified and sent in turn to seven observers (RM, AJC, MGC, JH, RZK, RAS, and HS). The observers are pathologists with an interest in dermatopathology and experience ranging from less than five years to several decades. Each observer was also sent guidelines (Fig. 1) containing definitions of the parameters to be assessed: dimensions of the SLN (Fig. 2a), intranodal location of tumor (subcapsular, parenchymal and/or sinusoidal, Fig. 2b), presence of extracapsular spread (ECS, Fig. 2c), tumor penetrative depth (TPD) of deposit(s) (maximum distance of melanoma cells from the inner margin of the SLN capsule, Fig. 2d), dimensions of the three largest deposits (or all deposits if fewer than three were present, eg Fig. 2e) and an estimate of percentage cross-sectional area of SLN occupied by metastatic melanoma, based on low power examination of the sections. The dimensions of the SLN and the deposits that were measured were a) the maximal dimension of the SLN/deposit along its long axis (dimension 1), and b) the maximal dimension of the SLN/deposit perpendicular to its long axis (dimension 2). All microscopic measurements were to be made with an ocular micrometer. A consensus microscopy session was not held at the commencement of the study, as the goal of the study was to determine the existing level of inter-observer variation in the evaluation of the various parameters in current practice.

Figure 1.

Guidelines for assessment.

Figure 2.

Histologic characteristics of SLN deposits.

a) Dimensions of SLN, d1 and d2, from which the cross sectional area of the SLN was calculated using the formula 0.25 × π × d1 × d2 (haematoxylin-eosin).

b) Location of melanoma in a lymph node - subcapsular (S), parenchymal (P) and/or sinusoidal (I). (HMB-45).

c) Extracapsular spread, i.e. tumor extension (ECS) outside the nodal capsule fom intranodal tumor.

d) Tumor penetrative depth of the deposit(s), the maximum subcapsular depth of tumor extension within the SLN (arrow). (S-100).

e) Deposit dimensions, d1 and d2, from which the cross sectional area of the deposit was calculated using the formula 0.25 × π × d1 × d2 for each deposit (S-100).

SLN tumor burden was calculated using only a microscope and an ocular micrometer. This involved utilizing the SLN dimensions and the dimensions of the three largest tumor deposits to determine the cross-sectional areas of the SLN and of the three largest deposits of melanoma, using the formula for the area of an ellipse [0.25 × π × dimension 1 × dimension 2]. The percentage cross-sectional area of the SLN occupied by metastatic melanoma was calculated from these figures. Although few SLNs and melanoma deposits in SLNs are precisely elliptical in cross-section, the method proposed provides a reasonable approximation of SLN tumor burden, while avoiding complex sectioning protocols and measurements, and the need for specialized computer/video-based morphometric equipment.

Statistical analysis was performed using SPSS 16.0 (SPSS Inc., Chicago IL, 2007). The level of inter-observer agreement was calculated using intraclass correlation coefficients (ICC). According to the guidelines of Landis and Koch,³⁷ ICC values of <0.20 can be interpreted as poor agreement, values of 0.21-0.40 as fair, values of 0.41-0.60 as moderate, values of 0.61-0.80 as good, and values of 0.81-1.00 as excellent agreement. The degree of correlation between the estimated and calculated percentage cross-sectional area of the SLN involved by tumor (intra-observer agreement) was calculated using the Pearson correlation coefficient. A p value of <0.05 was considered statistically significant.

RESULTS

The results are summarized in Table 2 and detailed in Table 3. There was good to excellent agreement between observers in the measurement of maximum size of the largest deposit (ICC = 0.88), the calculation of cross-sectional area of the SLN (ICC = 0.85) and of the three largest deposits (ICC = 0.73), the calculated and estimated percentage of the cross sectional area of the SLN occupied by the 3 largest deposits (ICC=0.68 and 0.94 respectively), and the TPD of metastases (ICC = 0.84). The level of agreement for the assessment of the subcapsular versus non-subcapsular location of metastases within the SLNs was moderate (ICS = 0.50), while that for the evaluation of extra-capsular spread ECS was fair (ICC = 0.39).

Table 2.

Interobserver agreement in the assessment of histologic parameters of metastatic melanoma deposits in SLNs

Characteristics	Cor relation score (95% confidence interval)	Degree of agreement (according to 37)
Quantitative parameters
Maximal size of largest SLN deposit	ICC = 0.88 (0.82-0.93)	Excellent
Cross-sectional area of SLN	ICC = 0.82 (0.74-0.89)	Excellent
Cross-sectional area of 3 largest deposits	ICC = 0.73 (0.63-0.82)	Good
Calculated % cross-sectional area occupied by 3 largest deposits	ICC = 0.68 (0.57-0.79)	Good
Estimated % cross-sectional area occupied by metastases	ICC = 0.94 (0.91-0.97)	Excellent
Tumor penetrative depth	ICC = 0.83 (0.76-0.90)	Excellent
Qualitative parameters
Location of deposits in SLN
Subcapsular vs non-subcapsular	ICC = 0.50 (0.37-0.64)	Moderate
Non-subcapsular locations
Parenchymal	ICC = 0.48 (0.35-0.63)	Moderate
Sinusoidal	ICC = 0.14 (0.06-0.26)	Poor
Extracapsular spread	ICC = 0.39 (0.27-0.54)	Fair

SLN — Sentinel lymph node; ICC — Intraclass correlation coefficient

Table 3.

Assessment of characteristics of metastatic melanoma deposits in SLNs by 7 observers: number of cases (% of number of cases scored)

	Observer 1	Observer 2	Observer 3	Observer 4	Observer 5	Observer 6	Observer 7
Maximal size of largest SLN deposit
N	44	39	44	44	41	42	43
<0.10mm	1 (2)	1 (3)	2 (5)	1 (2)	2 (5)	0 (0)	1 (2)
0.11-0.19mm	3 (7)	0 (0)	3 (7)	3 (7)	3 (7)	3 (7)	3 (7)
0.20-1.00mm	13 (30)	11 (28)	18 (41)	16 (36)	15 (37)	19 (45)	19 (44)
>1.00mm	27 (62)	27 (69)	21 (48)	24 (55)	21 (51)	20 (47)	20 (47)
Cross-sectional area of SLN
N	44	44	44	44	44	44	44
<10 mm2	4 (9)	3 (7)	4 (9)	5 (11)	4 (9)	10 (23)	3 (7)
>10-50 mm2	13 (30)	16 (36)	14 (32)	13 (30)	15 (34)	18 (41)	12 (27)
>50-100 mm2	11 (25)	7 (16)	12 (27)	11 (25)	13 (30)	7 (16)	11 (25)
>200 mm2	16 (36)	18 (41)	14 (32)	15 (34)	12 (27)	9 (20)	18 (41)
Cross-sectional area of 3 largest deposits
N	44	44	44	44	44	43	44
0.00-0.10 mm2	10 (23)	8 (18)	18 (41)	13 (30)	16 (36)	11 (26)	14 (32)
0.11-1.00 mm2	15 (34)	18 (41)	12 (27)	16 (36)	14 (32)	20 (47)	15 (34)
1.01-10.00 mm2	15 (34)	14 (32)	11 (25)	11 (25)	10 (23)	8 (19)	12 (27)
>10.00 mm2	4 (9)	4 (9)	3 (7)	4 (9)	4 (9)	4 (9)	3 (7)
Calculated percent cross-sectional area occupied by 3 largest deposits
N	44	44	44	44	44	43	44
0.00-1.00%	20 (45)	24 (55)	27 (61)	22 (50)	24 (55)	19 (44)	25 (57)
1.01-5.00%	11 (25)	8 (18)	8 (18)	11 (25)	9 (20)	11 (26)	9 (20)
5.01-10.00%	6 (14)	3 (7)	3 (7)	4 (9)	3 (7)	1 (2)	5 (11)
>10.00%	7 (16)	9 (20)	6 (14)	7 (16)	8 (18)	12 (28)	5 (11)
Estimated percent cross-sectional area occupied by metastases
N	44	43	44	44	41	44	43
0.00-1.00%	20 (45)	16 (37)	17 (39)	19 (43)	14 (34)	23 (52)	11 (26)
1.01-5.00%	13 (30)	13 (30)	17 (39)	14 (32)	11 (27)	12 (27)	17 (40)
5.01-10.00%	4 (9)	5 (12)	3 (7)	3 (7)	6 (15)	2 (5)	7 (16)
>10.00%	7 (16)	9 (21)	7 (16)	8 (18)	10 (24)	7 (16)	8 (19)
Tumor penetrative depth
N	43	42	44	44	41	44	43
0.00-0.29 mm	10 (23)	5 (12)	10 (23)	9 (20)	9 (22)	10 (23)	9 (21)
0.30-1.00 mm	16 (37)	15 (36)	12 (27)	13 (30)	13 (32)	20 (45)	14 (33)
>1.00 mm	17 (40)	22 (52)	22 (50)	22 (50)	19 (46)	14 (32)	20 (47)
Location
N	44	44	44	44	44	44	44
Subcapsular	40 (91)	33 (75)	37 (84)	39 (89)	30 (68)	39 (89)	32 (73)
Parenchymal	22 (50)	25 (57)	33 (75)	38 (86)	33 (75)	37 (84)	36 (82)
Sinusoidal	14 (32)	8 (18)	21 (48)	28 (64)	0 (0)	0 (0)	0 (0)
Capsular	4 (9)	11 (25)	3 (7)	15 (34)	3 (7)	1 (2)	8 (18)
Extracapsular spread
N	44	44	44	43	41	44	44
Present	2 (5)	4 (9)	0 (0)	3 (7)	0 (0)	3 (7)	1 (2)
Absent	42 (95)	40 (91)	44 (100)	40 (93)	41 (100)	41 (93)	43 (98)

N — total number of observations; NA — not applicable

The intra-observer agreement between each observer’s estimated and calculated values for the percent SLN area occupied by tumor deposits was good (median ICC = 0.78, mean ICC = 0.70, range 0.38-0.95). Among the four pathologists with greater than 5 years’ experience in dermatopathology, the agreement was similar (median ICC = 0.79, mean ICC = 0.72, range 0.51-0.81) to that among those with less than 5 years’ experience (median ICC = 0.70, mean ICC = 0.67, range 0.38-0.95).

DISCUSSION

Previous studies have shown that several characteristics of deposits of metastatic melanoma in SLNs correlate with the presence of tumor in non-SLNs in subsequent CLND specimens (Table 1). Parameters that were predictive in univariate analyses (and less so in multivariate analyses) include the location of tumor within SLNs,^{15, 20, 29} the TPD (centripetal thickness or maximum subcapsular depth),^{15, 24, 26, 30, 38-41} the presence of ECS,²² the presence of tumor in perinodal lymphatics,⁴⁰ and SLN tumor burden or size.^{15, 20, 21, 25-28, 39, 40} Many of these parameters also predict clinical outcome and provide important prognostic information. The size of SLN tumor deposits^{21, 28} and involvement of all three anatomic zones (subcapsular, parenchymal and sinusoidal areas) of SLNs by tumor²¹ are associated with a higher risk of recurrence. SLN tumor burden (size of SLN tumor deposits)^{17, 25, 28, 31} and TPD (in some studies^{30, 39}) are also significantly associated with survival. However, other studies of tumor in SLN using similar definitions have found no association between some histologic characteristics and CLND status or survival. For example, multivariate analysis in the study by Debarbieux et al²⁶ found size of the largest SLN metastasis to be predictive of disease-specific survival, while TPD and ECS were not found to be independent predictors of poorer disease-free survival. Frankel et al⁴² showed no association between the intranodal location of SLN deposits and CLND status. Govindarajan et al²¹ demonstrated that, although specific location of SLN tumor (in subcapsular, parenchymal or sinusoidal zones) did not predict the presence or absence of tumor in a CLND specimen, the presence of tumor in all three zones was independently associated with an increased rate of recurrence. Given these conflicting results, it remains unclear which characteristics of SLN tumor deposits most accurately predict tumor in non-SLN and provide the most accurate prognostic information.

Some of these SLN tumor characteristics have been clearly defined. TPD is defined as the maximum subcapsular depth of tumor within the SLN).^{15, 24, 26, 30, 38-40} ECS is the presence of tumor in perinodal tissues external to the lymph node capsule associated with the presence of tumor within the SLN.²² Location of tumor within lymph nodes has been characterized as subcapsular (in the subcapsular zone), parenchymal (within the nodal parenchyma), diffuse (extensive involvement of the SLN with effacement of nodal architecture) or multifocal.^{15, 20, 29} Tumor burden has been assessed differently in different studies using the Starz S-classification (tumor penetrative depth),^{15, 30, 38-40, 43} the largest size of tumor deposits,^{15, 17, 20, 21, 26, 27, 39, 40} or the cross-sectional area of such deposit(s),^28,40 estimated by pathologists, calculated based on micrometer measurements, or measured using computer-assisted morphometry.²⁵ In fact, nodal tumor deposits are often irregularly-shaped, may have ill-defined borders and their evaluation is in part dependent on sectioning protocols. More extensive sectioning may reveal additional tumor deposits or demonstrate a larger deposit(s) in the deeper sections (though peripheral tumor deposits are usually smaller).^{44, 45} Yet despite varying levels of precision of measurement, numerous studies have shown a positive correlation between the SLN tumor burden and tumor in non-SLN positivity and clinical outcome. It is apparent that the predictive value of SLN tumor burden holds regardless of the method of its assessment. Because of the variation of methods used to assess some histologic parameters, and the subjectivity inherent in the assessment of others, we sought to determine the degree of inter-observer variation in the assessment of these parameters.

We found that reproducibility of the assessment of TPD and SLN area was excellent, that inter-observer agreement in evaluation of the cross-sectional area of SLN deposits and calculation of the percentage of the SLN area involved by tumor was good. For each individual pathologist, estimation of the percentage of the SLN area involved by tumor using low power magnification was highly reproducible and closely comparable to the percentage calculated from painstaking and time consuming ocular micrometer measurements of deposits and SLNs. The results suggest that in experienced hands, estimation of tumor burden can apparently serve as an alternative to more labour-intensive and time-consuming methods. This finding should be confirmed in studies of larger patient groups, such as the Multicenter Sentinel Lymphadenectomy Trial II (MSLT-II).

Although the level of agreement for the measurement of maximal dimension of tumor was high (ICC = 0.88), we found that in occasional cases, a “shotgun scatter” pattern of metastatic melanoma was measured by one observer as a single large metastasis, while another/other observer(s) measured the tumor as several smaller adjacent deposits. It is because of the occurrence of such discrepancies that a more reliable measure of SLN tumor burden might be the percent area of SLN involvement, and the fact that estimated values of the latter parameter correlate well with the measured and calculated values makes this parameter particularly useful and reliable in the assessment of tumor burden.

Assessment of the location of tumor deposits within SLNs was less reproducible than the measurable parameters. It is generally recognized that assessment of the precise location of melanoma deposits within SLNs can be difficult. This is especially the case if tumor deposits are irregularly shaped or multifocal, and in routine histologic sections in which the different architectural compartments are not clearly distinguishable. In this study, agreement for assessment of subcapsular location of tumor compared with all non-subcapsular locations was moderate (ICC = 0.50). For the non-subcapsular locations, interobserver agreement was moderate for parenchymal sites (ICC = 0.48) and poor for sinusoidal sites (ICC = 0.14). The location of tumor in non-subcapsular locations has in general been shown to be more often associated with non-SLN involvement than tumor confined to subcapsular locations,²⁹ and therefore subcategorization of non-subcapsular tumor deposits seems unwarranted, particularly since such subcategorization is poorly reproducible. Providing a more precise definition of subcapsular location and having a consensual discussion of this would be likely to improve the reproducibility of its recording.

ECS is routinely reported by pathologists evaluating lymph node metastases and is a prognostically important observation.^46-48 ECS is rarely seen in SLNs containing metastatic melanoma, and we found that assessment of ECS was only fairly reproducible (ICC = 0.39). There are few reports of inter-observer agreement in the assessment of ECS in the literature.⁴⁹ Theunissen et al⁴⁹ showed only moderate inter-observer agreement (kappa = 0.50) in initial assessment of ECS in lymph nodes containing metastatic lung cancer. In our study, re-examination of the discrepant cases showed that the presence of ECS was difficult to assess due to difficulties presented by disruption of the nodal capsule overlying the tumor. Other reasons for poor reproducibility of ECS assessment include lack of ‘full-face’ sections, artifactual displacement of intranodal tumor cells into extranodal locations, and the difficulty in assessing tumor deposits associated with extensive fibrosis and desmoplasia. While the results suggest that reproducible assessment of ECS may be problematic, the numbers of cases showing ECS in this study are small, and further study of larger numbers of cases may clarify the issue.

Compared to the quantitative variables, the inter-observer agreement in assessment of location of tumor deposits in SLNs and ECS is relatively poorer. Clearer definitions and criteria may improve the reproducibility of assessment of these tumor characteristics. For example, Theunissen et al found that inter-observer reproducibility of the assessment of ECS in lymph nodes improved (kappa = 0.78) following the introduction of clear criteria for ECS.⁴⁹

The evaluation of characteristics of metastatic melanoma deposits in SLNs is clinically important for two reasons. Selected patients may be spared CLND on the basis of information derived from analysis of these parameters, for example patients with very small SLN tumor deposits.⁴³ Already in some centers, patients with SLN tumor deposits <0.1mm in maximal dimension are not offered CLND.³⁶ In this study, we found that only 2 cases were deemed by at least one observer to have a SLN deposit <0.1mm in size. However, 3 observers in one of these cases,and 4 observers in the other scored the deposit as being ≥0.1mm in size. Therefore clinical management of these patients according to the 0.1mm cutoff would have been different based on the pathologist evaluating the SLN. However, since only two cases fell within this size group in this study, it is difficult to draw definitive conclusions on this point. The results of MSLT-II, currently enrolling patients, will demonstrate whether or not it is appropriate to observe patients found to be SLN positive, rather than proceeding routinely to immediate CLND. In the future, staging and prediction of outcome in SLN-positive patients may be based on evaluation of characteristics of SLN tumor deposits, and future revisions of the American Joint Committee on Cancer staging system for melanoma will likely involve sub-staging of SLN based on differing outcomes associated with varying tumor loads.

Because the participating pathologists were not required to score each feature independently on each of the HE-stained and immunohistochemically stained sections, it was not possible to address the question of whether assessment of SLN parameters is more reproducible on HE-stained or immunohistochemically stained sections.

In summary, assessment of characteristics of metastatic melanoma in SLNs (such as TPD and tumor burden) is generally highly reproducible between pathologists in different institutions and with differing levels of experience. Estimated values of the percentage area of the SLN occupied by tumor correlated well with formally measured and calculated values of this parameter. This suggests that estimation of tumor burden may be sufficient, avoiding laborious and time-consuming measurement and calculation of nodal tumor burden. Validation studies are required to confirm the accuracy and reliability of the estimation method.