Trends in Accuracy and Comprehensiveness of Pathology Reports for Resected NSCLC in a High Mortality Area of the United States

Abstract

Introduction:

Complete and accurate pathology reports are vital to postoperative prognostication and management. We evaluated the impact of three interventions across a diverse group of hospitals on pathology reports of postresection NSCLC.

Methods:

We evaluated pathology reports for patients who underwent curative-intent surgical resection for NSCLC, at 11 institutions within four contiguous Dartmouth Hospital Referral Regions in Arkansas, Mississippi, and Tennessee from 2004 to 2020, for completeness and accuracy, before and after the following three quality improvement interventions: education (feedback to heighten awareness); synoptic reporting; and a lymph node specimen collection kit. We compared the proportion of pathology reports with the six most important items for postoperative management (specimen type, tumor size, histologic type, pathologic [p] T-category, pN-category, margin status) across the following six patient cohorts: preintervention control, postintervention with four different combinations of interventions, and a contemporaneous nonintervention external control.

Results:

In the postintervention era, the odds of reporting all key items were eight times higher than those in the preintervention era (OR = 8.3, 95 % confidence interval [CI]: 6.7–10.2, p < 0.0001). There were sixfold and eightfold increases in the odds of accurate pT- and pN-category reporting in the postintervention era compared with the preintervention era (pT OR = 5.7, 95 % CI: 4.7–6.9; pN OR = 8.0, 95 % CI: 6.5–10.0, both p < 0.0001). Within the intervention groups, the odds of reporting all six key items, accurate pT category, and accurate pN-category were highest in patients who received all three interventions.

Conclusions:

Gaps in the quality of NSCLC pathologic reportage can be identified, quantified, and corrected by rationally designed interventions.

Introduction

The 131,880 expected U.S. lung cancer deaths in 2021 will account for 22 % of all U.S. cancer mortality for both men and women.¹ The aggregate 5-year survival of the 235,760 individuals diagnosed with having lung cancer in the United States is estimated at 21 %.¹ Most long-term lung cancer survivors are patients with NSCLC who underwent curative-intent surgical resection. Nevertheless, surgery alone may not be curative, and certain high-risk patients require adjuvant treatment (chemotherapy, radiation therapy, targeted therapy, or their combinations).² Furthermore, high-risk patients are often selected for trials of novel adjuvant treatments.

Optimal postoperative management and prognostication depend on certain critical information from the pathologic evaluation of the resection specimen. Two College of American Pathologists (CAP) “Q-probes” studies revealed that a substantial proportion of post-operative pathology reports, across several cancer types, lacked important items, such as specimen type (anatomical extent of resection), margin status, extent of invasion, and status of lymph nodes.^3,4 Therefore, CAP and the American College of Surgeons’ Commission on Cancer (CoC) developed a checklist of items recommended for inclusion in pathology reports of lung cancer resection specimens from 2004 onward.^5,6 The use of synoptic reports, which contain specific fields for data insertion, was also recommended by the CAP as a means of standardizing reporting.⁶ This requirement is now explicitly stated in the American College of Surgeons CoC Operative Standard 5.8, which defines expectations for lung cancer surgery quality.⁷

Having previously reported deficiencies in the thoroughness and accuracy of pathology reporting on lung resection specimens, we evaluated the impact of the following three potentially corrective interventions: specific feedback to heighten awareness of benchmarked institutional performance, synoptic reporting, and a prelabeled lymph node specimen collection kit.^8,9

Materials and Methods

The Mid-South Quality of Surgical Resection Cohort

We constructed the Mid-South quality of surgical resection (MS-QSR) database, which includes detailed clinical information on patients who underwent curative-intent surgical resection for NSCLC in 11 institutions within four contiguous Dartmouth Hospital Referral Regions in East Arkansas, North Mississippi, and West Tennessee from 2004 to 2020. The MS-QSR is an ongoing population-based cohort, involving a diverse group of hospitals in this high U.S. lung cancer incidence and mortality region.^10,9 All data, including reported CAP checklist items from the final pathology report, were retrieved by trained data abstractors using a standardized template. The MS-QSR is systematically, independently, and periodically audited for accuracy.^10,9 All aspects of this study were conducted with oversight and approval from the institutional review boards of all participating hospitals and the University of Memphis.

CAP Checklist Items

Since January 2004, the minimum CAP and CoC-recommended specific reporting items for lung cancer resection specimens include specimen type (anatomical extent of resection), laterality, tumor site, greatest tumor dimension, histologic type, histologic grade, pathologic T-category, pathologic N-category, pathologic M-category, margin status, direct extension of tumor, venous (large vessel) invasion, arterial (large vessel) invasion, lymphatic (small vessel) invasion, and additional pathologic findings.⁷ Of these 15 items, we identified the following six with evidence-based direct impact on use of postoperative management as “crucial items”: specimen type, tumor size, histology, margin status, T-category, and N-category. We did not include M-category because, in most cases, clinicians have ready access to that information independently of the pathology report. Venous and arterial invasion items were merged into a single “vascular invasion” item for practical reasons because pathology reports did not sufficiently distinguish between these. In addition, we could not evaluate if pathologists reported the direct extension of a tumor in this analysis.

Accuracy of Reported pT- and pN-Categories

Pathologic T- and N-categories were independently determined for each patient by trained data abstractors, on the basis of a thorough review of all information included in the pathology report. These independent determinations were then audited by a second member of the research team. The accuracy of pathologists’ reporting of pathologic staging was determined by comparing the independently determined pT- and pN-categories, on the basis of all information in the pathology report, with the officially reported pT- and pN-categories at the top of the pathology report. TNM staging used the American Joint Committee on Cancer edition 6 from 2004 to 2009, edition 7 from 2010 to 2017, and edition 8 from 2018 to 2020. We converted all to edition 8 for combined reporting. Accurate pathologist reporting required that each TNM category was reported and matched the independently determined category.

Interventions

Our analysis of pathology reports from resections performed in Metropolitan Memphis hospitals from 2004 to 2008, included here as the preintervention (group 1) cohort, revealed a quality deficit in pathology reporting.⁸ From 2009 onward, at each participating hospital, we provided a general overview of the adverse survival implications of suboptimal pathologic nodal staging and specific, confidential feedback on institutional performance to key stakeholders (senior administrative leaders of all Metropolitan Memphis hospitals and the lung cancer surgeons and pathologists at those hospitals) by summarizing and systematically presenting the results of their baseline and subsequent lung resection pathology report quality (the educational intervention designed to heighten awareness). We also recommended (but could not enforce) adoption of synoptic reports with the CAP checklist items embedded and prospectively observed changes in pathology practice, documenting the dates of adoption by group. Synoptic reporting was adopted by some, but not all, pathology groups. Finally, we piloted a prelabeled lymph node specimen collection kit to improve hilar and mediastinal lymph node examination (from 2010 at certain hospitals), followed by a prospective staggered implementation study involving all hospitals from 2014 onward, as we have previously described.^9,11

Comparison Groups

On the basis of varying combinations of the three interventions, we created five groups of patients who had surgery in Metropolitan Memphis hospitals and a sixth, external control, group from surrounding non-Memphis hospitals (Table 1). We first evaluated results by era within the Memphis area hospitals, comparing all 2004 to 2009 patients (group 1) with all 2010 to 2020 patients (groups 2–5). We then evaluated differences in outcomes on the basis of the varying levels of intervention, within the four intervention groups, 2 to 5. Finally, the external control group included all lung cancer resections performed at six regional, nonmetropolitan Memphis area hospitals within the MS-QSR from 2010 to 2020. Cases included from these hospitals received none of the interventions; they are evaluated as external controls to delineate the impact of secular changes apart from the three interventions.

Table 1.

Characteristics of Six Cohorts of Patients From the Mid-South Quality of Surgical Resection Database 2004 to 2020

Characteristic	Group 1	Group 2	Group 3	Group 4	Group 5	Group 6a
N	1389	747	305	135	939	1243
Institutions	MMA	MMA	MMA	MMA	MMA	Non-MMA
Years	2004–2009	2010–2020	2010–2020	2010–2020	2010–2020	2010–2020
Educational intervention	No	Yes	Yes	Yes	Yes	No
Synoptic reporting	No	No	Yes	No	Yes	No
Kit use	No	No	No	Yes	Yes	No

^aExternal control group.

MMA, Memphis metropolitan area.

Statistical Analysis

We summarized demographic and clinical information for the entire cohort, stratified by intervention group and reported as mean and SD or frequency reported with percentages. Comparisons between intervention groups were made using the chi-square test or analysis of variance. Trends in yearly proportions were evaluated with the Cochran-Armitage test for trend. We fit logistic regression models and reported model-based ORs with 95 % confidence intervals (CIs). We report crude models and multiple variable models adjusted for surgeon and pathologist. In sensitivity analyses, we evaluated the potential impact of changing patient demographics on the changes in pathology reporting. These supplemental results include multiple variable models adjusted for age, sex, race, insurance, histology, and margin status and report adjusted ORs. In some cases, fewer variables were included in the models owing to nonconvergence of the full model. The p values less than 0.05 were considered statistically significant, and all analyses were conducted in Statistical Analysis System version 9.4 (Cary, NC).

Results

Demographic and Clinical Characteristics

The cohort consists of 4758 patients who underwent surgical resection for NSCLC from 2004 to 2020. The surgical operations were performed at 11 institutions (five in metropolitan Memphis, six nonmetropolitan) with pathologic evaluation by pathologists in seven pathology groups. The 1389 patients in the early era from 2004 to 2009 (group 1) were 56 % male with a median age of 67 years (interquartile range: 60–74), 75 % white, 24 % black, 7 % commercially insured, 65 % insured by Medicare, and 23 % insured by Medicaid (Supplementary Table 1). Most early era patients had adenocarcinoma (49 %) or squamous cell carcinoma (36 %). Of the patients, 90 % had pT1 or pT2 tumors and 70 % had pN0. Patients in the late era (2010–2020, groups 2–5) were more likely to be female, Medicare insured, have adenocarcinoma histology, undergo lobectomy or bilobectomy, and have negative resection margins (all p < 0.05; )Supplementary Table 1. There were slight differences in the pathologic stage distributions (pT, pN, pM, and aggregate) between the groups resulting in 3 % more stage I/II patients in the later era (p = 0.0021; Supplementary Table 1). Age and race distributions were significantly different among the six groups (p = 0.001 and 0.014 respectively, Supplementary Table 1).

Comprehensiveness and Accuracy of Reporting 2004 to 2009 Versus 2010 to 2020

From 2004 to 2009 (group 1), specimen type, laterality, tumor site, greatest tumor dimension, histologic type, histologic grade, and margin status were all reported 96 % of the time or greater. In the post-era (groups 2–5), the reporting of specimen type, tumor site, and greatest tumor dimension all improved significantly (p < 0.05; Table 2), whereas percentages reporting histologic type, histologic grade, laterality, and margin status all remained similarly high. Nevertheless, the completeness of reporting of pM decreased significantly over time, moving from 75 % to 17 % (p < 0.0001; Table 2). Reporting of all CAP checklist items decreased from 12 % to 3 % (p < 0.0001; assuming 100 % reporting of direct extension, which was not assessable in this study). When pM was not considered, complete reporting of all other checklist items did not differ from 2004 to 2009 versus 2010 to 2020 (13 % versus 14 %, p = 0.74; Table 2).

Table 2.

Comprehensiveness and Accuracy of Pathology Reports in 2004 to 2009 Versus 2010 to 2020

		Metro Institutions
Item Number	Item	2004–2009 (Group 1) n = 1389	2010–2020 (Groups 2–5) n = 2126	p Value	External Control n = 1243
1	Specimen typea	98.13	100.00	<0.0001	100.00
2	Laterality	99.57	99.86	0.1680	99.84
3	Tumor site	96.54	98.45	0.0002	98.55
4	Tumor greatest dimensiona	97.48	98.49	0.0315	99.03
5	Histologic typea	99.71	99.67	1.0000	99.84
6	Histologic grade	99.78	99.76	1.0000	99.60
7	pTa	69.91	96.38	<0.0001	88.74
8	pNa	68.47	95.48	<0.0001	87.45
9	pM	75.31	17.25	<0.0001	20.44
10	Margin statusa	97.26	98.68	0.0024	99.20
12–13	Vascular invasion	29.37	17.12	<0.0001	36.04
14	Lymphatic invasion	46.80	76.53	<0.0001	75.46
	All six key items	64.94	93.89	<0.0001	86.40
	All itemsb (with pM)	11.52	2.92	<0.0001	5.31
	All itemsb (without pM)	13.10	13.50	0.7353	31.05
	Accuracy of pT	66.59	91.86	<0.0001	84.63
	Accuracy of pN	67.75	94.40	<0.0001	86.40

^aIdentified as key item.

^bItem 15 (direct extension) is not available for analysis; aggregate measures consider it reported 100 %.

Metro, metropolitan.

During the preintervention era (2004–2009), complete reporting was high in four of the six checklist items we identified as crucial, which are as follows: specimen type (98 %), tumor site (97 %), histology (99.7 %), and margin status (97 %). Nevertheless, only 70 % reported pT, 68 % reported pN, and 65 % reported all six key items (Table 2). In the postintervention era (2010–2020), reporting of pT and pN improved significantly to 96 % and 95 %, respectively (both p < 0.0001; Table 2). Reporting of all six key items increased to 94 % in the postintervention era (p < 0.0001). When evaluated yearly, attainment of six key items continually increased until reaching a plateau approximately in 2014 (p < 0.0001 for trend; Fig. 1A).

Figure 1.

(A) Percentage of pathology reports with complete reporting of six key items by year from 2004 to 2020 (p < 0.0001 for trend). (B) Accuracy of pathology reporting of pT and pN by year from 2004 to 2020 (both p < 0.0001 for trend).

Reporting Accuracy

The accuracy of pT reporting (which requires both reporting by pathologist and concordance with independent assessment) was 67 % in the preintervention era (group 1) but improved to 92 % in the postintervention era (p < 0.0001). There was similar improvement in pN accuracy, moving from 68 % preintervention to 94 % postintervention (p < 0.0001). When evaluated yearly, the accuracy of reported pT and pN trended significantly upward (p < 0.0001 for trend; Fig. 1B).

Comprehensiveness and Accuracy Adjusted for Group Differences

Overall differences in accuracy and completeness between 2004 to 2009 and 2010 to 2020 were further compared with logistic regression models. In the postintervention era, the odds of reporting all six items were eight times higher than those in the preintervention era (OR 8.3, 95 % CI: 6.7–10.2, p < 0.0001). There were sixfold and eightfold increases in the odds of accurate pT (OR 5.7, 95 % CI: 4.7–6.9) and pN (OR 8.0, 95 % CI: 6.5–10.0) (both p < 0.0001) reporting in the postintervention era compared with the preintervention era. Results remained statistically significant in models adjusted for pathologist and surgeon (Table 3) and in sensitivity analyses adjusting for age, race, sex, insurance type, histology, and margin status (Supplementary Table 2).

Table 3.

Logistic Regression Models for Comprehensiveness and Accuracy of Pathology Reports

		Unadjusted		Adjusteda
Response Variables	Exposure Variables	OR (95 % CI)	p Value	OR (95 % CI)	p Value
All six key items	Group 1	Reference	—	Reference	—
	Groups 2–5	8.29 (6.73–10.2)	<0.0001	3.84 (2.83–5.21)	<0.0001
Accuracy of pT	Group 1	Reference	—	Reference	—
	Groups 2–5	5.66 (4.68–6.86)	<0.0001	2.26 (1.71–2.99)	<0.0001
Accuracy of pN	Group 1	Reference	—	Reference	—
	Groups 2–5	8.03 (6.47–9.97)	<0.0001	3.14 (2.30–4.29)	<0.0001
All six key items	Group 2	Reference	—	Reference	—
	Group 3	0.80 (0.51–1.26)	0.3346	0.93 (0.46–1.86)	0.8368
	Group 4	1.27 (0.61–2.62)	0.5218	1.40 (0.58–3.36)	0.4536
	Group 5	2.94 (1.86–4.65)	<0.0001	3.30 (1.61–6.80)	0.0012
Accuracy of pT	Group 2	Reference	—	Reference	—
	Group 3	1.39 (0.89–2.19)	0.1517	1.22 (0.64–2.33)	0.5462
	Group 4	1.39 (0.74–2.61)	0.3115	1.78 (0.82–3.86)	0.1428
	Group 5	2.69 (1.85–3.90)	<0.0001	1.87 (1.01–3.44)	0.0451
Accuracy of pN	Group 2	Reference	—	Reference	—
	Group 3	0.95 (0.59–1.52)	0.8256	1.05 (0.52–2.11)	0.8929
	Group 4	1.29 (0.63–2.66)	0.4912	1.24 (0.51–3.02)	0.6394
	Group 5	4.23 (2.54–7.07)	<0.0001	3.95 (1.87–8.33)	0.0003

^aControlling for pathologist and surgeon.

CI, confidence interval.

Comprehensiveness and Accuracy of Reporting by Intensity of Intervention

The postintervention era was further delineated on the basis of the level of intervention received (group 2: education only; group 3: education + synoptic reporting [on the basis of pathology group]; group 4: education + lymph node kit; group 5: education + synoptic reporting + lymph node kit; Table 1). Details of the completeness of reporting of each of the checklist items are found in Table 4. Reporting of all six key items improved from 92 % with education only (group 2) to % with all three interventions (group 5) (p < 0.0001; Table 4 and Fig. 2). The accuracy of reported pN also improved incrementally across the four intervention groups, from 92 % to 98 % (p < 0.0001; Table 4 and Fig. 2). Accuracy of reported pT also improved from 88 % to 95 %, (p < 0.0001; Table 4 and Fig. 2). Across all three of the most important metrics, the group with the combination of Education, Synoptic Reporting, and Lymph Node Kit had superior outcomes.

Table 4.

Comprehensiveness and Accuracy of Pathology Reports by Intensity of Intervention

		Metro Institutions From 2010 to 2020
Item Number	Item Education Synoptic reporting Kit use	Group 2 Yes No No n = 747	Group 3 Yes Yes No n = 305	Group 4 Yes No Yes n = 135	Group 5 Yes Yes Yes n = 939	p Value
1	Specimen typea	100.00	100.00	100.00	100.00	1.0000
2	Laterality	99.60	100.00	100.00	100.00	0.2146
3	Tumor site	97.19	98.69	97.04	99.57	0.0002
4	Tumor greatest dimensiona	98.53	98.36	97.78	98.62	0.7852
5	Histologic typea	99.46	98.67	99.26	99.89	0.1951
6	Hist. grade	99.46	100.0	100.0	99.89	0.2983
7	pTa	94.78	93.77	96.30	98.51	<0.0001
8	pNa	93.31	91.80	95.56	98.40	<0.0001
9	pM	16.87	22.11	3.70	17.95	<0.0001
10	Margin statusa	98.66	96.39	100.0	99.25	0.0033
12–13	Vascular invasion	18.21	16.72	2.96	18.42	0.0001
14	Lymphatic invasion	75.37	78.03	65.93	78.49	0.0097
	All six key items	91.70	89.84	93.33	97.02	<0.0001
	All itemsb (with pM)	2.01	3.61	0.00	3.83	0.0229
	All itemsb (without pM)	12.32	12.46	2.96	16.29	0.0001
	Accuracy of pT	88.09	91.15	91.11	95.21	<0.0001
	Accuracy of pN	91.57	91.15	93.33	97.87	<0.0001

^aIdentified as key item.

^bItem 15 (direct extension) is not available for analysis; aggregate measures consider it reported 100 %.

Metro, metropolitan.

Figure 2.

Attainment and accuracy of six key items, pT, and pN by intervention group.

We further evaluated differences in completeness and accuracy in logistic regression models. Odds of reporting all six key items seem higher in groups 4 and 5 compared with group 2, with group 5 rising to the level of statistical significance (OR = 2.9, 95 % CI: 1.9–4.7, p < 0.0001; Table 3). Accuracy of reported pT was higher in group 5 (OR = 2.7, 95 % CI: 1.9–3.9, p < 0.0001) compared with group 2 (Table 3), as was the accuracy of reported pN (OR = 4.23, 95 % CI: 2.5–7.1, p < 0.0001; Table 3). Results remained statistically significant in adjusted models (Table 3 and Supplementary Table 2).

External Control Group

The external control group consisted of 1243 cases from the modern era (2010–2020) that did not receive any of the three interventions, from institutions in the same region. We compared the three main quality measures (reporting all six key items, accuracy of reported pT, and accuracy of reported pN) in this group compared with groups 1 to 5. On all three measures, we found that the external control group had significantly better results than group 1 (all p < 0.0001) and significantly worse results than groups 2 to 5 combined (all p < 0.0001; Fig. 2).

Discussion

Because the pathology report provides the final word on a patient’s stage, it is critical that pathology reports are comprehensive (providing all the key elements needed for postoperative oncologic management) and accurate. Improvement in pathologic staging quality requires intervention in the following three domains: events during the surgical operation, communication between surgical and pathology teams, and events during the pathologic evaluation of resection specimens, up to and including the generation of the final report. Our evaluation of pathologic reporting for resected NSCLC revealed a serious quality gap during the baseline era, from 2004 to 2009, in this high lung cancer mortality region of the United States.⁷

Concurrently with an ongoing effort to improve surgical processes and the communication between operating room teams and pathology teams using a surgical lymph node specimen collection kit, we have also focused our quality improvement efforts on pathology teams.^9,12,13 These efforts include improvement in the thoroughness of gross dissection of resection specimens and improvements in the accuracy and comprehensiveness of reports using the CAP checklist items as the standard.^14,15

During the baseline period, from 2004 to 2009, we found great variability in the levels of completeness and accuracy of reporting for two key CAP checklist items, pT and pN, the major determinants of adjuvant therapy eligibility. Tumor size and margin status are also important prognostic factors that inform the selection of any postoperative treatment modality.^16,17 Histology influences the choice of adjuvant chemotherapy drug, and the specimen type (anatomical extent of resection) can influence the use of radiation therapy, which is more deleterious after pneumonectomy. Therefore, we specifically evaluated the reporting of these six key items in the patient cohorts.

After 2009, we found better reporting and accuracy of pT and pN in our external control group which received no intervention. The group that received education alone had better reporting and accuracy than the pre-era group and the external control group. The use of synoptic reporting provided an additional boost, as has been well revealed across multiple different cancers, including lung cancer.^18–21 Adoption of synoptic reports may vary between pathologists even within the same group; therefore, we adjusted results for pathologists.²² Finally, a lymph node collection kit provided additional benefit over and above that of education alone or education combined with synoptic reporting. The primary goal of the collection kit is to improve nodal sampling during surgical resection.^9,11 Nevertheless, a side benefit seems to be to improve the communication between the surgery and pathology teams and it seems to work synergistically with synoptic reporting.^23,24 This synergy is important, given the CoC Operative Standard 5.8 requirement for evaluation of lymph nodes from named or numbered hilar and mediastinal nodal stations and synoptic reporting.⁷

The main limitation of our study is the retrospective analytical design. The reporting of vascular invasion was not positively affected by any of the three interventions. In addition, the CAP recommendation for pathologists to report the pM category seems to be overwhelmingly disregarded in our community. This is probably because pathologists do not have reliable access to the M-category information, which most often is derived from radiologic studies, and is therefore best determined by the treatment team. Furthermore, in a surgical resection cohort, it can be assumed that most patients had M0 disease. Although we evaluated a large number of patients in a period of 12 years, the involvement of only 11 institutions, seven pathology groups, and 92 pathologists made it difficult to evaluate the influence of institutional and pathologist characteristics on the comprehensiveness and accuracy of pathology reportage. The predominance of community-level institutions and private practice pathologists raises questions on the relevance of our findings for patients who receive care at academic cancer care facilities. Nevertheless, 85 % of U.S. lung cancer surgery is performed in community health care systems, such as in the MS-QSR. Our study was limited to a single (albeit tristate) U.S. region, leaving open the possibility that some of the quality disparities observed are unique to our regional area. Nevertheless, we have previously revealed that the quality of care in this region is generalizable to the United States, and our location at the heart of the U.S. lung cancer incidence and mortality belt increases the public health relevance of our findings.^25,26

Complete and accurate pathology reports are vital to postoperative prognostication and management of patients with NSCLC. Existing gaps in the quality of pathologic reportage can be identified, quantified, and corrected by rationally designed interventions, which must then be implemented across the full diversity of environments where lung cancer care is delivered. The sustainability and survival impact of such pathology quality improvement interventions in diverse care environments and across a diversity of providers warrant further study.