Second opinion strategies in breast pathology: a decision analysis addressing over-treatment, under-treatment, and care costs

Abstract

Purpose

To estimate the potential near-term population impact of alternative second opinion breast biopsy pathology interpretation strategies.

Methods

Decision analysis examining 12-month outcomes of breast biopsy for nine breast pathology interpretation strategies in the U.S. health system. Diagnoses of 115 practicing pathologists in the Breast Pathology Study were compared to reference-standard-consensus diagnoses with and without second opinions. Interpretation strategies were defined by whether a second opinion was sought universally or selectively (e.g., 2nd opinion if invasive). Main outcomes were the expected proportion of concordant breast biopsy diagnoses, the proportion involving over-or under-interpretation, and cost of care in U.S. dollars within one-year of biopsy.

Results

Without a second opinion, 92.2% of biopsies received a concordant diagnosis. Concordance rates increased under all second opinion strategies, and the rate was highest (95.1%) and under-treatment lowest (2.6%) when all biopsies had second opinions. However, overtreatment was lowest when second opinions were sought selectively for initial diagnoses of invasive cancer, DCIS, or atypia (1.8 vs. 4.7% with no 2nd opinions). This strategy also had the lowest projected 12-month care costs ($5.907 billion vs. $6.049 billion with no 2nd opinions).

Conclusions

Second opinion strategies could lower overall care costs while reducing both over-and under-treatment. The most accurate cost-saving strategy required second opinions for initial diagnoses of invasive cancer, DCIS, or atypia.

Introduction

Increased awareness of diagnostic variation among pathologists has raised concerns that healthcare decisions based on inaccurate pathologic diagnosis could be compromised [1]. Recently, the Breast Pathology Study (B-Path) reported the extent of variation in pathologists’ diagnostic interpretation of breast biopsy tissue [2], and explored implications for individual women [3]. With overall concordance as low as 48% for atypia [2], there is clearly potential for improvement.

Appropriate clinical care is dependent on accurate pathologic diagnosis; diagnostic variation—a surrogate measure of accuracy—can lead to over-or under-interpretation. Absolute diagnostic accuracy is difficult to determine because true biologic standards remain elusive in clinical practice. Misclassification (over-or under-interpretation) directs patients to clinical care pathways that may result in over-or under-treatment and the attendant consequences. For pathologic interpretations of breast tissue, women diagnosed with more severe disease than they actually have may undergo unnecessary procedures and treatments, while women diagnosed with less severe disease may miss potentially life-saving interventions. Therefore, misclassification and misdiagnosis reduce the clinical effectiveness of breast cancer screening and increase potential harms to women.

Improving breast biopsy diagnostic interpretation accuracy is an important approach to enhancing quality of care and patient outcomes. Previously, we estimated the diagnostic accuracy of breast biopsy interpretations by practicing pathologists relative to expert consensus-reference diagnosis for invasive carcinoma, ductal carcinoma in situ (DCIS), atypia (atypical ductal hyperplasia [ADH]; ADH in a papilloma), and benign without atypia (proliferative and non-proliferative changes) [2]. These estimates were established using breast biopsy test sets designed to improve statistical confidence for diagnoses with low prevalence. We subsequently reported the clinical relevance of these estimates for individual women, showing that an estimated 4.6% of women receive more severe diagnoses and 3.2% receive less severe diagnoses compared with the reference-consensus diagnosis [3]. We also evaluated potential improvements in diagnostic accuracy using defined strategies for obtaining second opinions [4]. This paper builds on previous B-Path work by comparing second opinion strategies for pathologic breast tissue interpretation to determine the extent of over-and under-treatment and associated health care costs in the year following biopsy from a U.S. screening population perspective.

Methods

Decision analysis was used to estimate the effect of different second opinion strategies that could improve breast pathology diagnostic interpretation on over-and under-treatment and cost for the U.S. screening population. We estimated the expected probability that breast biopsy interpretations would agree with reference-standard diagnoses obtained in the B-Path Study under nine different strategies. We used interpretive performance of 115 pathologists participating in B-Path as a representation of diagnostic accuracy across the U.S [2, 3]. The probability that breast biopsy specimens within each reference-standard diagnosis would lead to over-or under-treatment was estimated based on standardized care pathways. Results were applied to the number of women undergoing breast biopsies each year in the U.S. to estimate the total numbers of women affected and the direct medical costs associated with each strategy in the year following a specific pathology diagnosis.

Breast pathology interpretation strategies

Strategies were defined based on the initiating trigger for a second opinion (Table 1), including: (1) the initial pathologist’s diagnostic interpretation (e.g., obtain a second opinion if the diagnosis is invasive cancer or DCIS); or (2) the pathologist’s desire for a second opinion for each case; (3) and/or each pathologist’s interpretation of the case relative to their laboratory’s current second opinion policy requirements (e.g., policy requires a second opinion for all invasive carcinoma diagnoses). The second opinion strategies did not require referral to centralized or expert pathologists, and are based on our prior evaluation of second opinion strategies [4]. While the percent of cases that will undergo secondary review differs under each strategy, we evaluate the impact that such strategies have on the 1-year outcomes for interpretation of all breast biopsy specimens—not only those requiring 2nd opinions.

Table 1.

The percent of cases requiring second (2nd) and third (3rd) reviews, misclassification rates (MCR), false positive (FP), and false negative (FN) proportions by pathology interpretation strategy and expert consensus diagnosis^*

Pathology interpretation strategy	Proliferative/non-prolif.		Reference standard expert consensus diagnosis
			Atypia		DCIS		Invasive
	Reviews 2nd, 3rd	MCR (FN, FP)	Reviews 2nd, 3rd	MCR (FN, FP)	Reviews 2nd, 3rd	MCR (FN, FP)	Reviews 2nd, 3rd	MCR (FN, FP)
No second opinion	0.0, 0.0	5.8 (0,5.8)	0.0, 0.0	52.2 (34.7,17.4)	0.0, 0.0	15.9 (13.3,2.6)	0.0, 0.0	3.9 (3.9,0)
Second opinion based on initial diagnosis
2nd if invasive	1.0, 0.8	5.4 (0,5.4)	0.4, 0.3	51.9 (34.4,17.5)	2.6, 2.4	13.6 (13.2,0.4)	96.1, 1.9	4.7 (4.7,0)
2nd if invasive or DCIS	3.2, 2.9	4.7 (0,4.7)	17.4, 13.1	43.7 (35.9,7.9)	86.7, 12.2	15.8 (15.2,0.6)	99.6, 3.3	3.9 (3.9,0)
2nd if invasive, DCIS or atypia	12.9, 10.4	2.1 (0,2.1)	65.3, 36.5	51.9 (41.8,10)	93.7, 17.8	12.1 (11.4,0.6)	99.6, 3.3	3.9 (3.9,0)
Second opinion based on pathologist desire or policy
2nd if desired	26.7,10.1	3.3 (0,3.3)	55.9, 31.3	47.7 (33.7,14)	30.5, 11.2	11.4 (9.9,1.5)	15.5, 1.5	3.6 (3.6,0)
2nd if considered borderline	19, 7.5	4.0 (0, 4.0)	45.3, 25.6	48.9 (34.6,14.2)	21.4, 9.3	11.5 (9.9,1.5)	3.5,0.8	3.8 (3.8,0)
2nd if required by policy	33.8, 7.6	4.7 (0,4.7)	40.6, 23	47.1 (33.9,13.2)	55.3, 10.0	13.7 (12.1,1.6)	59.9, 2.1	4.0 (4.0,0)
2nd if desired or required	54, 14.9	2.9 (0,2.9)	80.4, 45.3	44.3 (32.8,11.4)	75.5, 18.0	10.3 (9.5,0.9)	69.8, 3	3.7 (3.7,0)
Second opinion for all	100, 19.7	2.9 (0,2.9)	100, 55.9	40.9 (29.9,11.1)	100, 21.6	9.9 (9.3,0.6)	100, 3.7	3.5 (3.5,0)

^*Second opinion strategies require resolution by the third reviewer if the first two disagree

Decision-analytic model and assumptions

The model evaluated each strategy based on a common underlying distribution of reference-standard diagnoses in four diagnostic categories: (1) invasive carcinoma; (2) DCIS; (3) atypia, which comprised ADH; and (4) benign pathology with or without proliferative changes (Fig. 1). Age-specific diagnosis distributions were those observed among women undergoing needle core or excisional breast biopsy following a screening mammogram using data from the Breast Cancer Surveillance Consortium (BCSC) over the period 2004 through 2008 (See Table 3 in Appendix).

Fig. 1.

Schematic representation of interpretation strategies depicting a common distribution of reference-standard diagnoses for each interpretation strategy. Final diagnostic interpretation and ensuing clinical outcomes are affected by the interpretation strategy’s misclassification rate relative to the reference-standard diagnosis as well as conditional probabilities of over and under-interpretation when a diagnostic misclassification occurs. *The probability of over-interpretation is zero for women with reference-standard diagnosis of invasive cancer. ^**The probability of under-interpretation is zero for women with reference-standard diagnosis of other benign findings

For each strategy, the probability of obtaining a final pathology interpretation that was misclassified relative to the reference-standard diagnosis was estimated from B-Path [2] (Table 1). Reference-standard diagnoses in B-Path were based on a consensus of three experienced breast pathologists who independently interpreted a case, followed by a consensus meeting. B-Path misclassification estimates were applied to the prevalence of categorical in the U.S. population to estimate the expected probability that each woman’s diagnosis was concordant with the reference-standard diagnosis for each strategy.

A diagnosis hierarchy was used to establish over-and under-treatment, where the most extensive treatment was for invasive cancer, followed by DCIS, then atypia and then benign/non-proliferative findings without atypia. Over-treatment was defined as treatment for the diagnosis received relative to the reference-standard diagnosis. Under-treatment was considered to occur when the woman’s diagnosis was lower in the treatment hierarchy than the reference-standard diagnosis (e.g., atypia diagnosis when DCIS was the reference-standard diagnosis).

We assumed that each woman’s care pathway would be based on her final pathological diagnosis according to current clinical guidelines [5]. Women receiving an initial diagnosis of invasive cancer were assumed to receive invasive cancer care regardless of their true underlying disease state. Likewise, women with an initial DCIS diagnosis were assumed to receive care and associated costs for DCIS care regardless of true diagnosis. Women with an initial diagnosis of atypia were assumed to undergo an open surgical biopsy. On the basis of the open surgical biopsy, we assumed that the true underlying disease state would be recognized and that disease-concordant care would be provided. For women with atypia as the true underlying diagnosis, we assumed no further costs within the year under baseline assumptions. However, under alternative assumptions of more aggressive care, women with atypia were assumed to undergo breast MRI and treatment with hormonal therapy. Women with other benign diagnoses were assumed to undergo only a mammography within the subsequent year.

The expected care costs were modeled under each alternative interpretation strategy using 2013 U.S. dollars. Medicare national fee schedules were used to estimate the cost associated with each procedure and included physician and technical reimbursement amounts. To ascertain the direct medical care costs of each care pathway, costs of breast imaging, additional pathological interpretations, core biopsies, excisional biopsies, and treatment costs for DCIS or invasive cancer in the year following such diagnoses were included in the model based on estimates from published studies [6, 7] (see Table 3 in Appendix).

Population projections were made by applying the estimated probabilities of concordant diagnoses, over-interpretation and under-interpretation, and expected costs to the number of women undergoing breast biopsies in the U.S. each year. We estimated the population of women undergoing diagnostic breast biopsy based on 2013 population counts from U.S. Census data [8], the 2013 National Health Interview Survey (NHIS) [9], and breast biopsy rates [10]. Based on these data sources, we estimated that between 738,000 and 843,000 diagnostic breast biopsies are done in the U.S. each year, and we assumed a base case number of 790,500 women undergoing breast biopsies each year.

Estimated savings

To estimate the potential savings associated with alternative strategies, we compared the total care costs for each strategy with care costs when second opinions were obtained according to policy requirements as reported by B-Path participants. The proportion of participating pathologists reporting that they would ask for a second opinion based on their laboratory’s policy was 33% for benign without atypia, 40% for atypia (ADH), 58% for DCIS, and 60% for invasive cancer diagnoses [11]. We report the incremental savings that could be achieved over and above current policy requirements for obtaining a second opinion for each strategy that resulted in cost savings. Lower and upper bounds for the estimated savings were based on the 95% confidence interval for the misclassification rates in B-Path.

Sensitivity analyses

We undertook sensitivity analyses to ascertain the effect of different assumptions and model parameters on the estimated savings associated with second opinion strategies. In one analysis, we modified care pathways to include more intensive care involving use of breast MRI and hormonal therapy with tamoxifen following an atypia diagnosis. In other analyses, we considered the effects that doubling the cost of obtaining second opinions or increasing the number of women affected each year to 1.6 million, a commonly cited figure for annual U.S. breast biopsies [12].

Results

Based on assumptions in the model, in the absence of a second opinion for breast pathology interpretations, 92.2% of breast biopsies would receive a reference-standard-concordant diagnosis resulting in a projected 729,078 concordant diagnoses in the U.S. each year (Table 2). Concordant diagnoses can be considered clinical estimates of diagnostic accuracy (i.e., correct diagnoses). Misclassified diagnoses, estimated at 61,422 each year, can be considered clinical estimates of the total number of breast biopsies with an incorrect diagnosis. The number of misclassified diagnoses decreased under all second opinion strategies evaluated in the model. Misclassification was lowest when either all biopsies were subjected to a second opinion (N = 39,051 with misclassified diagnoses), or when second opinions were obtained only when the initial pathologists’ diagnosis was invasive cancer, DCIS, or atypia (N = 39,683). When pathologists desired a second opinion or policy requirements called for one, misclassifications were similar in magnitude (N = 40,316).

Table 2.

Expected annual number of women with concordant and misclassified diagnoses and expected costs of care in the year following biopsy by strategy

Pathology interpretation strategy	Number of women per year						Care costs in year following diagnosis ($ in billions)
	Concordant diagnoses	Lower bound	Upper bound	Misclassified diagnoses	Lower bound	Upper bound	Costs	Lower bound	Upper bound
No second opinion	729,078	715,403	743,149	61,422	47,351	75,098	$6.049	$6.018	$6.084
Second opinion based on initial diagnosis
2nd if invasive	731,292	710,185	749,552	59,208	40,948	80,315	$5.976	$5.894	$6.023
2nd if invasive or DCIS	737,141	717,695	753,505	53,359	36,995	72,805	$5.913	$5.817	$5.971
2nd if invasive, DCIS or atypia	750,817	735,165	763,307	39,683	27,193	55,335	$5.907	$5.804	$5.973
Second opinion based on pathologist’s desire or current policy
2nd if desired	746,390	726,944	759,038	44,110	31,462	61,422	$5.992	$5.930	$6.023
2nd if considered borderline	741,963	723,070	760,935	48,537	32,806	67,430	$5.980	$5.902	$6.019
2nd if required by policy	737,457	729,078	760,935	53,043	36,126	72,884	$6.036	$5.968	$6067
2nd if desired or required	750,185	717,616	754,374	40,316	26,640	56,758	$6.002	$5.929	$6.045
Second opinion for all	751,449	735,165	764,809	39,051	25,691	55,335	$6.060	$5.992	$6.098

Lower and upper bounds are based on 95% confidence interval for misclassification rate

The improvement in diagnostic accuracy with the addition of second opinion strategies was accompanied by reduced rates of over-treatment relative to under-treatment (Fig. 2). As depicted in Fig. 2, the number of women experiencing over-treatment was lowest when second opinions were sought selectively for cases initially diagnosed with invasive cancer, DCIS, or atypia.

Fig. 2.

Estimated number of cases misclassified by diagnostic interpretation strategy with number of over-and under-treated shown for each pathologic interpretation strategy

Under base case assumptions, the no second opinion strategy resulted in breast-related direct medical costs of $6.049 billion in the year following the breast biopsy diagnosis (Table 2). In comparison, costs were lowest at $5.907 billion when second opinions were sought for cases initially interpreted as invasive, DCIS, or atypia—an estimated savings of $142 million per year with improved outcomes for potentially over 20,000 women.

Many pathology laboratories in the US have policies requiring that a second opinion be obtained on specific cases, usually new diagnoses of invasive cancer. Accuracy improved for over 17,000 women (misclassified diagnoses dropped to 44,110) when these policies were analyzed in the model. However, when the cost savings of improved treatment were considered, the care costs were similar to results of strategies that considered not obtaining second opinions at all ($6.036 billion/year vs. $6.049 billion/year with no second opinions, respectively).

Despite adding costs for second interpretations in the model, all second opinion strategies projected overall savings except for the strategy of obtaining a second opinion for all breast biopsies each year (Table 2). Estimated cost savings were greatest for the strategy that triggered a second opinion when the initial pathologist reported that invasive cancer, DCIS, or atypia was present with an expected annual savings of $142 million per year relative to no second opinion or of $129 million per year relative to current policy requirements. As analytic assumptions were varied in sensitivity analyses (Fig. 3), projected cost savings for this strategy remained high, increasing to an expected annual savings of over $250,000,000 per year if the actual total number of breast biopsies is 1.6 million per year as has been estimated by some [12]. When the cost of obtaining a second opinion was doubled (from base case of $140 to $280), the strategy of obtaining a second opinion if desired or required by current policies no longer saved money under lower bound misclassification estimates.

Fig. 3.

Expected annual savings for specified strategies relative to current policy requirements for second opinion as base case assumptions are varied. Bars reflect the lower and upper bound estimates (1) Title of Y axis (‘‘Relative to if a second opinion was required by policy’’; and on X Axis: (2) Basecase Analyses (‘‘Based on 790,500 women undergoing breast biopsies every year according to baseline care pathways (see Appendix)’’; and (3) More Intensive Treatment Pathways (‘‘Based on 790,500 women undergoing breast biopsies every year but with breast MRI and hormonal therapy used in the year following a breast diagnosis involving Atypia’’)

Discussion

Our decision analysis evaluated second opinion strategies for pathologic interpretation of breast tissue to determine the extent of over-and under-treatment and associated costs of health care in the year following biopsy from a U.S. screening population perspective. Our population projections indicate that the vast majority of women who undergo breast biopsy each year in the United States receive pathological interpretations that are concordant with consensus expert opinion diagnoses. While this is reassuring [3], our analyses also suggest that there is potential for improvement in pathologic diagnosis. Compared with current second opinion policies, apart from a strategy of second opinion for all biopsy interpretations, the highest number of additional concordant diagnoses was projected when second opinions were triggered by an initial diagnosis of invasive cancer, DCIS, or atypia. Under that strategy, an additional 13,360 women were projected to receive concordant diagnoses. Importantly, that strategy was also estimated to result in a lower expenditure of healthcare resources relative to current policies for second opinions. Overall, our analysis suggests that annual potential savings of approximately $129 million could be realized if second opinions were routinely sought following an initial pathology interpretation of invasive cancer, DCIS, or atypia.

When compared with current policy requirements for obtaining second opinions, we found that several strategies may reduce overall expenditures on breast-related care in the year following a biopsy. However, it is important to consider the balance of women projected to receive over-versus under-treatment under each strategy. The lowest costs were estimated for second opinions triggered by initial interpretations of invasive cancer, DCIS, or atypia, but this strategy had relatively more under-versus over-treated women. While this suggests a lower chance of over-diagnosis, our analysis did not directly address the issue of over-diagnosis, which is defined as the diagnosis of a dis-ease that will not come to medical attention or adversely affect an individual during his or her lifetime [13]. Given the 1-year time horizon for our analysis, we were not able to evaluate over-diagnosis, which is challenging to evaluate in both clinical and model-based studies [14, 15].

There is growing awareness that diagnoses of DCIS may not all pose a threat to a woman’s expected survival and that active surveillance may be a reasonable course of action [16]. Our study used established care guidelines and care pathways to estimate costs of care in the year following the biopsy and, therefore, did not consider the option of active surveillance which may be less costly. Estimates of costs associated with over-diagnosis in the year following a DCIS diagnosis may be lower in settings of active surveillance.

To demonstrate the value of obtaining second opinions for outside pathology diagnoses referred to a tertiary care setting, one study evaluated all pathology cases in a single month and reported the frequency of diagnostic discrepancies that may have affected care [17]. Among 297 outside breast cases, there were 40 (13.47%) discrepancies, with 14 (4.73%) characterized as potentially significant. Of the 14 cases, 8 were documented to have changes in individual clinical management costs ranging from a savings of $90,642 to an added cost of $115,832. Another study from the same institution focused on breast pathology cases for which second opinions were sought and found significant discrepancies in 226/1970 cases (11.47%) [18]. While intraductal lesions were identified among the most problematic cases, this study also included pathology diagnoses not considered in B-Path (lobular carcinoma, metaplastic carcinomas, and phyllodes tumors), which were also noted as often being discrepant.

For second opinion strategies to succeed, it is critical that health systems support pathologists in obtaining second opinions. The B-Path study showed that nearly all participating pathologists (96%) believed second opinions improve diagnostic accuracy and that many seek second opinions even in the absence of institutional policies [11]. The manpower demands for obtaining second opinions were not considered in our study, but it is possible that second opinions could be obtained from within the same institution. With the growth of telepathology, it is increasingly possible that second opinions could be efficiently instituted through centralized services [18, 19]. Time lags in the diagnostic evaluation can be stressful for women, and digital imaging might help to speed up this process. Digital whole slide imaging was approved by the FDA in 2017 for primary diagnosis.

While our results suggest potential cost savings for second opinion strategies, these findings must be interpreted with caution due to several limitations inherent in our analysis. First and foremost, our population projections were based on a simulation study of second opinion strategies and involved a limited set of slides and only a single slide per case. We also did not consider the expertise of the pathologists from whom a second opinion was sought. However, changes in overall misclassification rates have been shown to differ little by expertise [4].

We conclude that relative to current second opinion policies, strategies that require breast biopsy specimens initially read as invasive cancer, DCIS, or atypia have the potential to improve diagnostic accuracy for more than 13,000 women each year, while also reducing both over-and under-treatment, and lowering overall care costs by an estimated $129 million per year. Implementation of second opinion strategies in clinical settings should be studied to evaluate efficient best practices for improving pathologic interpretation of breast biopsy specimens.

Appendix

See Table 3.

Table 3.

Model parameters, data sources and assumptions

Pathology diagnosis distribution from the Breast Cancer Surveillance Consortium (BCSC)a	Invasive	DCIS	Atypia	Benign
Ages 40–49	11.0%	4.9%	3.2%	80.9%
Ages 50–59	20.2%	8.5%	4.3%	67.1%
Ages 60–69	29.6%	8.8%	3.1%	58.4%
Ages 70–79	35.9%	9.2%	3.0%	51.9%
Ages ≥80	47.0%	11.3%	3.1%	38.6%

Cost components in 2013 U.S. Dollars	Costs	Source	Assumptions
Treatment for DCIS within year following diagnosis	$13,376.08	Yabroff (2008) [7]
Treatment for invasive carcinoma within year following diagnosis	$27,043.02	Yabroff (2008) [7]	Prevalence-weighted care costs for regional and distant breast cancer
Hormone therapyd	$1469.65		Average wholesale price for Tamoxifen 20 mg/day
‘‘Follow-up’’ digital mammogram	$277.62	Medicare	Digital screening mammogram with associated professional and technical fees
Bilateral screening breast MRI	$1103.34	Medicare
Core breast biopsyb	$936.90	Plevritis (2006) [6]	Ultrasound-guided core needle biopsy and stereotactic biopsy, weighted based on their respective probabilities
Open surgical biopsy	$1657.39
Second opinion breast pathology interpretationc	$140.17	Medicare	Tissue exam by pathologist with associated professional and technical fees

^aBased on observed breast pathology outcomes among women 504,032 women undergoing 994,085 screening mammograms in 2004 through 2008

^bThe cost of core breast biopsy is a combined value from the costs of ultrasound-guided core needle biopsy and stereotactic biopsy, weighted based on their respective probabilities

^cThe cost of core biopsy’s additional reading is the summation of costs from the CPT codes for a tissue exam by pathologist and the associated professional and technical components

^dBased on average wholesale price for Tamoxifen given at a regimen of 20 mg/day