Abstract
Background
Audits of operative summaries and pathology reports reveal wide discordance in identifying the extent of lymphadenectomy performed (the communication gap). We tested the ability of a pre-labeled lymph node specimen collection kit and checklist to narrow the communication gap between operating surgeons, pathologists, and auditors of surgeons’ operation notes.
Methods
We conducted a prospective single cohort study of lung cancer resections performed with a lymph node collection kit from November 2010 to January 2013. We used the kappa statistic to compare surgeon claims on a checklist of lymph node stations harvested intraoperatively, to pathology reports, and an independent audit of surgeons’ operative summaries. Lymph node collection procedures were classified into 4 groups based on the anatomic origin of resected lymph nodes: mediastinal lymph node dissection, systematic sampling, random sampling and no sampling.
Results
From the pathology report, 73% of 160 resections had a mediastinal lymph node dissection or systematic sampling procedure, 27% had random sampling. The concordance with surgeon claims was 80% (kappa statistic 0.69 [CI 0.60 – 0.79]). Concordance between independent audits of the operation notes and either the pathology report (kappa 0.14 [0.04 – 0.23]), or surgeon claims (kappa 0.09 [0.03 – 0.22]), was poor.
Conclusion
A pre-labeled specimen collection kit and checklist significantly narrowed the communication gap between surgeons and pathologists in identifying the extent of lymphadenectomy. Audit of surgeons’ operation notes did not accurately reflect the procedure performed, bringing its value for quality improvement work into question.
Introduction
Accurate pathologic nodal staging, a powerful prognostic factor after resection of lung cancer, requires thorough examination of the mediastinal lymph nodes. Mediastinal lymph node dissection or systematic sampling are recommended [1,2], but infrequently performed [3–6]. For example, 62% of pathologic N0 and N1 non-small cell lung cancer resections in the US Surveillance, Epidemiology, and End Results database have no mediastinal lymph nodes examined [6]. Accurate pathologic staging of lung cancer requires effective collaboration between the surgical and pathology teams.
There is significant discordance between surgeon claims of the extent of mediastinal lymphadenectomy and verifiable lymph node examination from pathology reports. In one city-wide audit, although 45% of all resections were claimed by operating surgeons to have had a ‘mediastinal lymph node dissection’, none met objective criteria using mediastinal nodal stations identified in the pathology report. Review of pathology reports suggested that only 8% of all resections met systematic sampling criteria, 50% had random sampling, and 42% had no mediastinal lymph nodes examined. However, a blinded independent audit of the surgeons’ operation notes suggested that 29% of cases had described a mediastinal lymph node dissection procedure [7].
These observations suggest that the quality gap in pathologic nodal staging may arise from 3 sources: poor surgical lymph node examination practice (failure to collect nodes) [6,7], problems in the transfer of specimens (loss in transit or improper communication of the source of specimens) [8], and poor pathology examination practice (incomplete examination or inaccurate reportage) [9]. The discordance between observers of the pattern of mediastinal lymph node examination, which has been described as a ‘Tower of Babel,’ poses a major problem for quality improvement efforts [10].
Pre-labeled lymph node specimen collection kits significantly improve the surgical collection of mediastinal lymph nodes and the quality of nodal staging [11–13]. How these kits influence the consistency of identification of mediastinal dissection procedure by different observers is unclear. We therefore examined the concordance rate of mediastinal lymph node examination procedures identified by different observers in a series of curative-intent lung cancer resections performed with a specially designed surgical lymph node specimen collection kit.
Material and Methods
With the approval of Institutional Review Boards of all participating institutions, including a waiver of the informed consent requirement for this low-risk quality improvement project, we conducted a prospective single cohort study of lung cancer resections performed with a pre-labeled surgical lymph node specimen collection kit at 4 community hospitals in Memphis, TN from November 2010 to January 2013. Three of the 4 hospitals were teaching hospitals, with surgery residents. Eligible cases had curative-intent surgical resection of lung cancer, without prior chemotherapy or radiation therapy.
The operating surgeons were all board certified cardiothoracic surgeons. Surgeons, operating room nursing staff, and pathology staff received training on the value of rigorous mediastinal lymph node examination and proper use of the specimen collection kit before provision of the kit for use at each institution. The kit, which has been described in detail elsewhere [12], included a standardized checklist on which a member of the operating room team identified the lymph node stations harvested during the operation. Surgeon claims of extent of mediastinal lymphadenectomy were extrapolated from the stations noted on the checklist as having been harvested. The anatomic stations identified in the pathology report provided the pathologist’s perspective of the mediastinal lymphadenectomy procedure.
Independent audit
We sent the same redacted copies of surgeons’ operative summary notes to an experienced academic general thoracic surgeon at 2 different National Cancer Institute-designated Comprehensive Cancer Centers for independent audit. These auditors were blinded to all records, except the redacted surgeons’ operation notes (which were all dictated free-text notes performed without a standardized institutional template), and performed their initial audits independently. All discordant audits were resolved during a single conference call discussion between the independent surgeons in order to provide a consensus audit. As an indirect, and unannounced, means of judging the thoroughness of the auditors, we also requested determination of the extent of resection performed, since this was more likely to be readily described in the operation notes. Auditors were asked to assign each resection into one of 4 possible categories: pneumonectomy, lobectomy/bilobectomy, segmentectomy or wedge resection.
Identification of nodal staging procedures
The mediastinal lymph node examination procedures were classified into 4 pre-defined groups based on stations marked on the checklist (surgeon claims), and the pathology report. Using the American College of Surgeons Oncology Group (ACOSOG) Z0030 trial definition, a mediastinal lymph node dissection was defined as including a minimum of stations 2R,4R, 7, 8, 9 and 10R for right side resections, and stations 4L, 5, 6, 7, 8, 9, and 10L for left sided dissections. A systematic sampling procedure did not meet mediastinal lymph node dissection criteria, but at a minimum, had lymph nodes from stations 4R, 7, and 10R on the right and 5,6,7 and 10L on the left [14]. Resections with one or more mediastinal lymph nodes, which did not meet the definitions of mediastinal lymph node dissection or systematic sampling were identified as random sampling and those with no mediastinal lymph nodes were identified as no samping. All operation notes were classified by the independent surgeon auditors into one of the 4 mediastinal lymph node examination categories by interpretation of the narrative description of the operation.
Statistical methods
The primary endpoints were the verifiable rate of mediastinal lymph node dissection and systematic sampling procedures by the 3 different types of observers (operating surgeon, pathologist, and auditing surgeon), and the concordance rate between observers. We used descriptive statistics to summarize patient demographic and clinical characteristics; calculated the concordance rate between observers by cross-matching the specific mediastinal nodal examination procedures in 2×2 format; and performed a formal statistical analysis of the comparison between observers with the kappa and weighted kappa statistic [15]. We reanalyzed the data in the lobectomy only population. Because the findings were very similar, we have only reported results for the whole cohort.
In grading the degree of concordance between different observers with the Kappa statistic, 0.00 – 0.20 was poor, 0.21 – 0.4 was slight, 0.41 – 0.6 was moderate, 0.61 – 0.8 was substantial, and >0.8 was almost perfect concordance [16].
Results
During the time course of our study from November 2010 to January 2013, 160 patients received curative-intent surgical resection in which a lymph node specimen collection kit was used. The demographic and clinical characteristics of these patients are summarized on Table 1, including details of their surgical experience, and the extent of the nodal dissection according to the pathologist’s report. Seventy three percent of all resections met ACOSOG Z0030 criteria for mediastinal lymph node dissection or systematic sampling and 27% had random sampling. All participants had at least 1 mediastinal lymph node examined. A median of 6 mediastinal lymph nodes were examined, collected from a median of 5 mediastinal nodal stations.
Table 1.
Cohort demographic and clinical characteristics in relation to objective review of extent of lymph node examination according to the pathology report.
| Variable | Surgical Lymph Node Procedure from Pathology Report, N (%)
|
|||
|---|---|---|---|---|
| All N=160 |
MLND N=72 (45) |
SS N=45 (28) |
RS N=43 (27) |
|
|
| ||||
| Age, median (IQR) | 67 (60 – 75) | 67 (63 – 77) | 68 (58 – 73) | 63 (58 – 74) |
|
| ||||
| Sex | ||||
| Female | 82 (51) | 38 (53) | 24 (53) | 20 (46) |
| Male | 78 (49) | 34 (47) | 21 (47) | 23 (54) |
|
| ||||
| Race | ||||
| Black | 34 (21) | 11 (15) | 11 (24) | 12 (28) |
| White | 124 (78) | 59 (82) | 34 (76) | 31 (72) |
| Other | 2 (1) | 2 (3) | 0 (0) | 0 (0) |
|
| ||||
| Health insurance | ||||
| Commercial | 93 (58) | 37 (51) | 27 (60) | 29 (67) |
| Medicaid | 17 (11) | 12 (17) | 2 (5) | 3 (7) |
| Medicare | 40 (25) | 20 (28) | 11 (24) | 9 (21) |
| None | 10 (6) | 3 (4) | 5 (11) | 2 (5) |
|
| ||||
| Histology | ||||
| Adenocarcinoma | 95 (59) | 41 (57) | 29 (64) | 25 (58) |
| Squamous | 50 (31) | 23 (32) | 11 (25) | 16 (37) |
| Other | 15 (9) | 8 (11) | 5 (11) | 2 (5) |
|
| ||||
| T-category | ||||
| 1 | 63 (39) | 27 (38) | 17 (38) | 19 (44) |
| 2 | 71 (44) | 35 (49) | 19 (42) | 17 (39) |
| 3 | 20 (13) | 9 (13) | 6 (13) | 5 (12) |
| 4 | 4 (3) | 1 (1) | 1 (2) | 2 (5) |
| X | 2 (1) | 0 (0) | 2 (4) | 0 (0) |
|
| ||||
| N-category | ||||
| 0 | 116 (73) | 55 (76) | 29 (64) | 32 (74) |
| 1 | 25 (16) | 10 (14) | 7 (16) | 8 (19) |
| 2 | 19 (12) | 7 (10) | 9 (20) | 3 (7) |
|
| ||||
| Extent of resection | ||||
| Pneumonectomy | 12 (8) | 7 (10) | 2 (4) | 3 (7) |
| Lobe/bilobectomy | 141 (88) | 63 (87) | 43 (96) | 35 (81) |
| Sublobar | 7 (4) | 2 (3) | 0 (0) | 5 (12) |
|
| ||||
| Resection technique | ||||
| Open | 117 (73) | 45 (62) | 37 (82) | 35 (81) |
| Robotic-assisted | 28 (18) | 22 (31) | 1 (2) | 5 (12) |
| Video-assisted | 15 (9) | 5 (7) | 7 (16) | 3 (7) |
|
| ||||
| Number of nodes, median (IQR) | ||||
| N1 | 5 (2 – 9) | 5 (2 – 10) | 5 (3 – 6) | 5 (2 – 7) |
| N2 | 6 (5 – 10) | 8 (6 – 11) | 6 (4 – 8) | 4 (3 – 7) |
| Total | 13 (9 – 17) | 15 (11 – 18) | 10 (8 – 15) | 11 (7 – 15) |
|
| ||||
| Number of N2 stations, median (IQR) | 5 (3 – 5) | 5 (5 – 6) | 4 (3 – 4) | 3 (2 – 4) |
|
| ||||
| ≥12 mediastinal lymph nodes, n (%) | 21 (13) | 13 (18) | 4 (9) | 4 (9) |
MLND= mediastinal lymph node dissection, SS= systematic sampling, RS= random sampling, NS= no sampling, IQR= interquartile range
Concordance between pathology report and operating surgeon claims
Based on the checklist, surgeons indicated they performed a mediastinal nodal dissection or systematic sampling procedure in 71% of cases, random sampling in 25%. The checklist had no information in 6 cases (4%), which the study team interpreted as having had no sampling. However, in 3 of these cases the pathology report indicated a mediastinal lymph node dissection had been performed, the other 3 had systematic sampling (Table 2). The concordance rate between the operating surgeon claims and the pathology report was 80% (kappa score 0.69 [95% confidence interval 0.6 – 0.79], indicating substantial agreement between observers).
Table 2.
Concordance between pathology report and surgeon reports.
| Mediastinal lymph node procedure from pathology report, N (%) | Number of mediastinal nodes | Number of mediastinal nodal stations | |||
|---|---|---|---|---|---|
| MLND, 72 (45) | SS, 45 (28) | RS 43 (27) | Median (interquartile range) | ||
|
| |||||
| Surgeon claims | |||||
| MLND, 79 (49) | 65 (41) | 7 (4) | 7 (4) | 8 (6 – 10) | 5 (5 – 6) |
| SS, 35 (22) | 0 (0) | 31 (19) | 4 (3) | 5 (4 – 9) | 4 (3 – 4) |
| RS, 40 (25) | 4 (3) | 4 (3) | 32 (20) | 5 (3 – 8) | 3 (2 – 5) |
| NS, 6 (4) | 3 (2) | 3 (2) | 0 (0) | 8 (8 – 11) | 5 (4 – 6) |
| Concordance rate | 80% | ||||
| Kappa (95% CI) | 0.69 (0.60 – 0.79) | ||||
| Weighted Kappa (95% CI) | 0.73 (0.64 – 0.82) | ||||
| Consensus audit | |||||
| MLND, 121 (76) | 62 (39) | 31 (19) | 28 (18) | 6 (5 – 10) | 5 (4 – 5) |
| SS, 22 (14) | 5 (3) | 9 (6) | 8 (5) | 7 (4 – 8) | 4 (3 – 5) |
| RS, 12 (8) | 3 (2) | 3 (2) | 6 (4) | 5 (3 – 9) | 3 (2 – 5) |
| NS, 5 (3) | 2 (3) | 2 (2) | 1 (1) | 7 (6 – 7) | 4 (3 – 6) |
| Concordance rate | 48% | ||||
| Kappa (95% CI) | 0.14 (0.04 – 0.23) | ||||
| Weighted Kappa (95% CI) | 0.16 (0.05 – 0.27) | ||||
Independent audits
Auditor 1 identified 77% of cases as mediastinal lymph node dissection, 11% as systematic sampling, and 4% as no sampling, compared to auditor 2 who identified 65% of cases as mediastinal lymph node dissection, 13% as systematic sampling, and 8% as no sampling (Table 3). The concordance rate between the 2 auditors was 74%, (kappa score 0.44 [CI 0.33 – 0.55], indicating moderate agreement between observers). After discussion of all cases between the auditors, the consensus was that 76% of operation notes had described a mediastinal lymph node dissection, 14% systematic sampling, and 3% no sampling (Table 4). Auditor 1 was closer to the consensus (kappa score 0.84, [CI 0.75 – 0.93], almost perfect agreement) than auditor 2 (kappa score 0.54, [CI 0.43 – 0.65], moderate agreement).
Table 3.
Concordance between auditors.
| Mediastinal lymph node procedure determined by Auditor 1, N (%)
|
||||
|---|---|---|---|---|
| MLND, 123 (76.9) | SS, 18 (11.3) | RS, 12 (7.5) | NS, 7 (4.4) | |
|
| ||||
| Auditor 2, N (%) | ||||
| MLND, 104 (65.0) | 102 (63.8) | 2 (1.3) | 0 (0.0) | 0 (0.0) |
| SS, 21 (13.1) | 11 (6.9) | 6 (3.8) | 3 (1.9) | 1 (0.6) |
| RS, 23 (14.4) | 6 (3.8) | 9 (5.6) | 6 (3.8) | 2 (1.3) |
| NS, 12 (7.5) | 4 (2.5) | 1 (0.6) | 3 (1.9) | 4 (2.5) |
| Concordance rate | 74% | |||
| Kappa (95% CI) | 0.44 (0.33 – 0.55) | |||
| Weighted Kappa (95% CI) | 0.52 (0.40 – 0.64) | |||
Table 4.
Comparison of individual auditors’ report to consensus audit.
| Mediastinal lymph node procedure determined from Consensus Audit, N (%)
|
||||
|---|---|---|---|---|
| MLND, 121 (75.6) | SS, 22 (13.8) | RS, 12 (7.5) | NS, 5 (3.1) | |
|
| ||||
| Auditor 1, N (%) | ||||
| MLND, 123 (76.9) | 119 (74.4) | 4 (2.5) | 0 (0.0) | 0 (0.0) |
| SS, 18 (11.3) | 1 (0.6) | 16 (10.0) | 1 (0.6) | 0 (0.0) |
| RS, 12 (7.5) | 1 (0.6) | 1 (0.6) | 10 (6.3) | 0 (0.0) |
| NS, 7 (4.4) | 0 (0.0) | 1 (0.6) | 1 (0.6) | 5 (3.1) |
| Concordance rate | 94% | |||
| Kappa (95% CI) | 0.84 (0.75 – 0.93) | |||
| Weighted Kappa (95% CI) | 0.85 (0.75 – 0.94) | |||
|
|
||||
| Auditor 2, N (%) | ||||
| MLND, 104 (65.0) | 103 (64.4) | 1 (0.6) | 0 (0.0) | 0 (0.0) |
| SS, 21 (13.1) | 9 (5.6) | 10 (6.3) | 1 (0.6) | 1 (0.6) |
| RS, 23 (14.4) | 5 (3.1) | 10 (6.3) | 8 (5.0) | 0 (0.0) |
| NS, 12 (7.5) | 4 (2.5) | 1 (0.6) | 3 (1.9) | 4 (2.5) |
| Concordance rate | 78% | |||
| Kappa (95% CI) | 0.54 (0.43 – 0.65) | |||
| Weighted Kappa (95% CI) | 0.62 (0.51 – 0.74) | |||
The concordance rate between the consensus audit and the pathology report was 48% (kappa 0.14 [CI 0.04 – 0.23], poor agreement) (Table 2). As an internal check on the thoroughness of the independent auditors’ review of the narrative operation notes, we also asked them to determine the extent of surgical resection (Table 5). The concordance rate between the pathology report and the consensus audit of extent of resection was 97% (kappa 0.84 [CI 0.71 – 0.98], almost perfect agreement).
Table 5.
Concordance on extent of surgical resection between auditors and pathology report.
| Extent of resection determined from Pathologist report, N (%) | ||||
|---|---|---|---|---|
|
| ||||
| Lobe/bilobectomy 143 (89.4) | Pneumonectomy 11 (6.9) | Segmentectomy 1 (0.6) | Wedge 5 (3.1) | |
|
|
||||
| Auditor 1, N (%) | ||||
| Lobe/bilobectomy 146 (91.3) | 140 (87.5) | 3 (1.9) | 0 (0.0) | 2 (1.2) |
| Pneumonectomy 10 (6.3) | 2 (1.3) | 8 (5.0) | 0 (0.0) | 0 (0.0) |
| Segmentectomy 2 (1.3) | 1 (0.6) | 0 (0.0) | 1 (0.6) | 0 (0.0) |
| Wedge resection 2 (1.3) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 3 (1.9) |
| Concordance rate | 94% | |||
| Kappa (95% CI) | 0.33 (0.24 – 0.41) | |||
| Weighted Kappa (95% CI) | 0.46 (0.33 – 0.58) | |||
|
| ||||
| Auditor 2, N (%) | ||||
| Lobe/bilobectomy 144 (90.0) | 139 (88.1) | 2 (1.3) | 0 (0.0) | 1 (0.6) |
| Pneumonectomy 11 (6.9) | 1 (0.6) | 9 (5.6) | 0 (0.0) | 1 (0.6) |
| Segmentectomy 2 (1.3) | 1 (0.6) | 0 (0.0) | 1 (0.6) | 0 (0.0) |
| Wedge resection 3 (1.9) | 2 (0.0) | 0 (0.0) | 0 (0.0) | 3 (1.9) |
| Concordance rate | 96% | |||
| Kappa (95% CI) | 0.38 (0.28 – 0.48) | |||
| Weighted Kappa (95% CI) | 0.54 (0.42 – 0.65) | |||
|
| ||||
| Consensus Audit, N (%) | ||||
| Lobe/bilobectomy 142 (88.8) | 140 (87.5) | 1 (0.6) | 0 (0.0) | 1 (0.6) |
| Pneumonectomy 11 (6.9) | 1 (0.6) | 10 (6.3) | 0 (0.0) | 0 (0.0) |
| Segmentectomy 3 (1.9) | 2 (1.3) | 0 (0.0) | 1 (0.6) | 0 (0.0) |
| Wedge resection 4 (2.5) | 0 (0.0) | 0 (0.0) | 0 (0.0) | 4 (2.5) |
| Concordance rate | 97% | |||
| Kappa (95% CI) | 0.84 (0.71 – 0.98) | |||
| Weighted Kappa (95% CI) | 0.82 (0.66 – 0.98) | |||
Finally, we tested the number of mediastinal lymph nodes and lymph node stations examined in concordant and discordant cases between the surgeons’ claims and the independent audits (Table 6). Cases in which the operating surgeon claimed a mediastinal lymph node dissection or systematic sampling procedure had a significantly higher number of mediastinal lymph nodes and stations examined than cases in which the operating surgeon claimed random sampling, irrespective of the auditors’ assignments. In all 6 cases in which the surgeon did not fill out the checklist, the operation notes were deemed to have described a mediastinal lymph node dissection according to the consensus audits (Table 6).
Table 6.
Comparison of the number of lymph nodes in pathology reports of concordant and discordant cases between operating and auditing surgeons.
| Mediastinal lymph node retrieval according to procedure | ||||
|---|---|---|---|---|
| Observer
|
N (%) | Nodes* | Stations* | |
| Surgeon’s claim | Consensus audit | |||
| MLND/SS | MLND | 104 (65.0) | 7 (5 – 10) | 5 (5 – 6) |
| MLND/SS | RS | 6 (3.8) | 8 (5 – 10) | 5 (4 – 6) |
| MLND/SS | NS | 4 (2.5) | 7 (7 – 10) | 7 (6 – 7) |
| RS | MLND | 33 (20.6) | 5 (3 – 8) | 4 (2 – 5) |
| RS | NS | 1 (0.6) | 4 (4 – 4) | 3 (3 – 3) |
| RS | RS | 6 (3.8) | 3 (3 – 4) | 2 (2 – 3) |
| NS | MLND | 6 (3.8) | 8 (8 – 11) | 5 (4 – 6) |
| Concordance rate | 69% | |||
| Kappa (95% CI) | 0.09 (0.03 – 0.22) | |||
| Weighted Kappa (95% CI) | 0.10 (0.04 – 0.24) | |||
Median (interquartile range).
Comment
Despite its prognostic and post-operative management decision-making value, multiple reports describe the existence of a major gap between optimal pathologic nodal staging and actual clinical practice [3–6, 17]. A lymph node specimen collection kit, which includes a checklist to remind the operating room team of the optimal lymph node dissection procedure required, has consistently improved the quality of pathologic staging, significantly narrowing the nodal staging quality gap [12,13]. In the current study, we evaluated the impact of the kit and checklist on the communication gap between surgeons and pathologists in reporting the extent of mediastinal lymphadenectomy.
Previously, we found that although surgeons claimed to have performed a mediastinal lymph node dissection in 45%, random sampling in 8%, and no sampling in 48% of cases in a large community-based cohort, no cases met ACOSOG mediastinal lymph node dissection criteria, and only 8% met systematic sampling criteria. Fifty percent of resections were random sampling and 42% had no sampling of mediastinal lymph nodes [7]. Having already demonstrated that use of this kit eliminates the practice of lung cancer surgery without mediastinal nodal sampling, we were interested in evaluating the impact of the kit on the clarity of communicating the extent of mediastinal lymphadenectomy, as indicated by the concordance between different observers- the operating surgeon, pathologist, and independent auditors.
Our primary interest was the concordance between operating surgeons and the pathologists, but we were also interested in examining the impact on independent auditors’ interpretation of the narrative operation summary notes. The reason for this secondary interest is that quality improvement imposes a mandate for accurate audits. Accurate audits require accurate source documentation, without which there can be no foundation for quality improvement [10]. Use of the checklist to indicate specific stations from which specimens have been collected significantly improved the concordance rate between surgeon claims and the pathology report from 39% in our previous study to 80% in this study. The kappa statistic, 0.69, suggests substantial agreement between the surgeons’ checklist and the pathology report. The distribution of number of sampled lymph nodes and nodal stations supports this concordance.
Concordance between the auditors, who relied solely on the operation notes, and either the pathology report, or the operating surgeons’ checklist-based claims was poor (kappa scores of 0.14 and 0.09 respectively). This is a problem for quality improvement studies that rely on independent audits of surgeons’ operation notes. Possible explanations for this persistent communication gap include poorly written operation notes and poor interpretation of the operation narrative by the auditors. There was moderate concordance between the auditors in determining the mediastinal sampling procedure (kappa 0.44), although auditor 1 was closer to the consensus audit than auditor 2, suggesting a difference in the thoroughness of review or accuracy of interpretation of the narrative documents between the two auditors.
However, the high level of consensus between the auditors and the pathology reports in identifying the extent of surgical resection suggests that both auditors reviewed the operation notes thoroughly, but had difficulty interpreting the narrative of the mediastinal dissection procedure. The number of lymph nodes and nodal stations examined was more closely associated with the operating surgeons’ claims than the auditor’s report, supporting the idea that the operating surgeons’ claim, obtained from the checklist, was more reliable than the interpretation of the operation notes. This bears implications for future surgical quality improvement studies, suggesting that interpretations based on the operation notes may over- and underestimate the extent of nodal dissection actually performed. Checklists perform better [18].
We chose a non-comparative study design for this report because we were interested in testing corrective interventions in real-world populations, having previously reported the enormity of the quality gap in pathologic nodal staging and the existence of a communication gap between surgeons and pathologists. The non-comparative study design, however, precludes determination of causality between the improvements we have described, and the intervention we introduced (the kit with a checklist). It is possible that unrelated events, such as heightened awareness of the importance of pathologic nodal staging, and increased oversight of the quality of nodal staging, account for the improvements we have described. However, we have previously reported how resections performed without the kit in the era after it had been introduced regressed in quality of pathologic nodal staging back to the era before it was introduced [12].
It is not certain that the kit and checklist intervention will be uniformly successful with different types of surgeons, in different types of institutions. This question is the subject of an ongoing tri-state ‘dissemination and implementation’ study involving disparate surgeons (cardiothoracic and general surgeons) operating in a diversity of institutions (rural, suburban and urban) in a region of the US with some of the highest lung cancer incidence and mortality rates. Preliminary results suggest the possibility of significant improvement in the quality of communication of nodal mapping between surgeons and pathologists after lung resection.
Additional work still needs to be done to improve the quality of the narrative description of mediastinal lymphadenectomy in surgeons’ operation notes. A checklist indicating nodal stations from which specimens were collected provides more accurate information. This bears significant implications for the design of future quality improvement initiatives.
Acknowledgments
Sharon Phillips for initial assistance with statistics.
This study was supported by NIH R01CA172253 (Osarogiagbon).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Darling GE, Allen MS, Decker PA, Ballman K, et al. Randomized trial of mediastinal lymph node sampling versus complete lymphadenectomy during pulmonary resection in the patient with N0 or N1 (less than hilar) non-small cell carcinoma: results of the American College of Surgery Oncology Group Z0030 Trial. J Thorac Cardiovasc Surg. 2011;141:662–70. doi: 10.1016/j.jtcvs.2010.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Howington JA, Blum MG, Chang AC, Balekian AA, Murthy SC. Treatment of stage I and II non-small cell lung cancer: Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest. 2013;143(5 Suppl):e278S–313S. doi: 10.1378/chest.12-2359. [DOI] [PubMed] [Google Scholar]
- 3.Little AG, Rusch VW, Bonner JA. Patterns of surgical care of lung cancer patients. Ann Thorac Surg. 2005;80:2051–6. doi: 10.1016/j.athoracsur.2005.06.071. [DOI] [PubMed] [Google Scholar]
- 4.Varlotto JM, Recht A, Nikolov M, Flickinger JC, Decamp MM. Extent of lymphadenectomy and outcome for patients with stage I nonsmall cell lung cancer. Cancer. 2009;11:851–8. doi: 10.1002/cncr.23985. [DOI] [PubMed] [Google Scholar]
- 5.Verhagen AF, Schoenmakers MC, Barendregt W, Smit H, van Boven WJ, Looijen M, van der Heijden EH, van Swieten HA. Completeness of lung cancer surgery: is mediastinal dissection common practice? Eur J Cardiothorac Surg. 2012;4:834–8. doi: 10.1093/ejcts/ezr059. [DOI] [PubMed] [Google Scholar]
- 6.Osarogiagbon RU, Yu X. Mediastinal lymph node examination and survival in resected early-stage non-small-cell lung cancer in the surveillance, epidemiology, and end results database. J Thorac Oncol. 2012;7:1798–806. doi: 10.1097/JTO.0b013e31827457db. [DOI] [PubMed] [Google Scholar]
- 7.Osarogiagbon RU, Allen JW, Farooq A, Wu JT. Objective review of mediastinal lymph node examination in a lung cancer resection cohort. J Thorac Oncol. 2012;7:390–6. doi: 10.1097/JTO.0b013e31823e5e2d. [DOI] [PubMed] [Google Scholar]
- 8.Makary MA, Epstein J, Pronovost PJ, Millman EA, Hartmann EC, Freischlag J. Surgical specimen identification errors: A new measure of quality in surgical care. Surgery. 2007;141:450–5. doi: 10.1016/j.surg.2006.08.018. [DOI] [PubMed] [Google Scholar]
- 9.Ramirez RA, Wang CG, Miller LE, Adair CA, Berry A, Yu X, O’Brien TF, Osarogiagbon RU. Incomplete intrapulmonary lymph node retrieval after routine pathologic examination of resected lung cancer. J Clin Oncol. 2012;30:2823–8. doi: 10.1200/JCO.2011.39.2589. [DOI] [PubMed] [Google Scholar]
- 10.Detterbeck FC. The fable of Babel and building a foundation for quality. J Thorac Oncol. 2012;7:267–8. doi: 10.1097/JTO.0b013e318243262a. [DOI] [PubMed] [Google Scholar]
- 11.Molnar TF. A new device for the identification of lymph nodes at lung cancer surgery. Eur J Cardiothorac Surg. 2007;31:311–312. doi: 10.1016/j.ejcts.2006.11.047. [DOI] [PubMed] [Google Scholar]
- 12.Osarogiagbon RU, Miller LE, Ramirez RA, et al. Use of a surgical specimen-collection kit to improve mediastinal lymph-node examination of resectable lung cancer. J Thorac Oncol. 2012;7:1276–82. doi: 10.1097/JTO.0b013e318257fbe5. [DOI] [PubMed] [Google Scholar]
- 13.Osarogiagbon RU, Ramirez RA, Wang CG, et al. Dual intervention to improve pathologic staging of resectable lung cancer. Ann Thorac Surg. 2013;96:1975–81. doi: 10.1016/j.athoracsur.2013.07.009. [DOI] [PubMed] [Google Scholar]
- 14.Allen MS, Darling GE, Pechet TT, et al. Morbidity and mortality of major pulmonary resections in patients with early-stage lung cancer: initial results of the randomized, prospective ACOSOG Z0030 trial. Ann Thorac Surg. 2006;81:1013–9. doi: 10.1016/j.athoracsur.2005.06.066. [DOI] [PubMed] [Google Scholar]
- 15.Szklo M, Nieto FJ. Epidemiology: Beyond the Basics. 3. Burlington, MA: Jones and Bartlett Learning; 2012. [Google Scholar]
- 16.Lawal B. Measures of agreement. In: Lawal B, editor. Categorical Data Analysis with Sas and Spss Applications. 1. Mahwah, NJ: Lawrence Erlbaum Associates, Inc; 2003. pp. 484–489. [Google Scholar]
- 17.Osarogiagbon RU, Yu X. Nonexamination of lymph nodes and survival after resection of non-small cell lung cancer. Ann Thorac Surg. 2013;96:1178–89. doi: 10.1016/j.athoracsur.2013.05.021. [DOI] [PubMed] [Google Scholar]
- 18.Gawande A. The checklist: if something so simple can transform intensive care, what else can it do? New Yorker. 2007 Dec 10;:86–101. [PubMed] [Google Scholar]