Interval cancer audit and disclosure in breast screening programmes: An international survey

Abstract

Objective and Setting

Accurate monitoring of interval cancers is important both for quality improvement and education and is a key parameter of breast screening quality assurance. Issues in relation to communication regarding interval cervical cancer in the Irish cervical screening programme were found, prompting interval cancer process review in all cancer screening programmes. An international survey to examine international consensus on interval breast cancer audit processes was conducted to inform Irish processes.

Methods

A survey of 24 international population-based breast screening programmes was done to determine which undertook audit of interval breast cancer; if yes, they were asked (1) how they undertake audit, (2) if they obtain individual consent for audit and inform women of audit results, and (3) if disclosure of audit results occurs.

Results

Response was 71% (17/24). Of these, 71% (12/17) have a programmatic audit process to calculate the interval cancer rate (ICR). Of these, ten also carry out radiological reviews, three using a blinded review. Two inform patients that audit is taking place; two provide choice to be in the audit; nine state that routine screening consent covers audit. For two of the five that have an open disclosure policy for medical incidents, this policy applies to screening interval cancers. One other country/region has an open disclosure policy for category 3 interval cancers only. Five have legal protection for interval cancers arising in the screened population.

Conclusion

While consistency in providing aggregate programmatic audits exists, there is no consistent approach to individual interval cancer reviews or results disclosure.

Introduction

In breast screening, interval cancers are defined “as breast cancers that arise after a negative screening episode (which may include assessment) and before the next scheduled screening round”. Interval cancers are typically classified into three categories after radiological review: Category 1 (normal/benign); Category 2 (uncertain, minimal signs); Category 3 (suspicious findings/false negative/missed). ¹ Interval cancers are not unexpected in population-based screening programmes. They are known to have a relatively worse prognosis than screen-detected cancers. The accurate monitoring of interval cancers is important both for quality improvement and education. The European Commission Initiative on Breast Cancer monitoring and evaluation subgroup recently included the interval cancer rate (ICR) as one of 13 key indicators of breast screening performance. ² Muratov et al. noted that the ICR is a direct measure of screening sensitivity which may be influenced by underlying incidence and the screening interval. The European Reference Organisation for Quality Assured Breast Screening and Diagnostic Services (EUREF) also requires programmes that apply for accreditation to calculate a programmatic ICR. ³ ICR calculations are based on proportional incidence of interval breast cancers against pre-screening incidence rates. ⁴ The risk of interval cancer increases with time after the index screen; Hofvind et al. found 70% of interval cancers occurring in the second year following the index screen in the Norwegian screening programme. ⁵

Apart from ICR calculations, individual radiological reviews may be carried out and the index mammogram classified retrospectively. Such reviews may be prompted by a) teaching and radiology training, b) patient or physician request, or c) as part of an internal systematic review process. The main difficulty with retrospective reviews is hindsight bias. Hindsight bias is a term used to explain the tendency of people to overestimate their ability to have predicted an outcome that could not possibly have been predicted. In other words, once it is known that a woman has an interval cancer it is almost impossible to review any previous mammograms without this knowledge influencing (even subconsciously) the readers’ interpretation and classification of that study. Hindsight bias is the key reason why there can be such wide intra and inter observer variations in the proportion of cases assigned false negative and false positive on review. The difficulty with reproducibility with such a categorisation is one of the reasons why sub-classification is not consistently utilised as a quality indicator in population screening programmes while the overall ICR is regarded as critical.

Individual reviews may be conducted under controlled conditions to minimise hindsight bias, where the review process attempts to replicate routine screen-reading circumstances, where the prevalence of breast cancer is extremely low, and where mammograms are read in batches over a short period of time. ⁶ An example of this is blinding of reviews. Blinded reviews are performed where the interval cancer mammograms are seeded among routine mammograms (controls), for review by radiologists who are unaware they are reviewing interval cancers; or the interval cancer mammograms are included in a batch of routine screening mammograms, but the radiologist knows there is an interval cancer among the mammograms. Blinded reviews can be difficult to set up, are time consuming, and in many cases are not essential when reviewing anonymised cases such as for teaching or training or for individual patient requested information. In these situations the interval cancer mammogram may be reviewed initially by the screening radiologist without the diagnostic study, knowing it is an interval cancer, and then compared directly with the diagnostic study side by side.

A strong communications strategy about the limitations of screening and how audit and individual review outcomes are communicated to relevant stakeholders is fundamental to all population screening programmes. A non-blinded, non-anonymised systematic review of individual interval cervical cancer cases in Ireland in 2018 raised concerns about communications relating to this review in the Irish CervicalCheck programme ⁷ and this prompted an expert evaluation of the management of interval cancers in all the Irish cancer screening programmes. Following this, in 2019 the Irish Health Services Executive established an Expert Reference Group (ERG) for each of the cancer screening programmes to define future audit and disclosure of interval cancers in Irish screening programmes. The terms of reference of the ERG included “definition of the future audit processes and review guidance for interval cancer based on international evidence and best practice”. ⁸ The aim of this study was to determine if consensus exists in interval cancer audit practice across population breast screening programmes worldwide.

Methods

A survey of international population-based breast screening programmes was performed in 2019 to determine if they undertook audit of interval breast cancers. Those countries/regions that did perform audit were asked (1) to describe how the audit was undertaken, including how the reviews were performed and how they controlled for retrospective bias, (2) to describe how consent for the audit was obtained and how women were informed of the audit process, and (3) to describe how the results of the audits were disclosed to patients.

The inclusion criteria for the survey were: (1) a national or regional population-based breast screening programme; (2) a country or region with a population ≥ population of Ireland; (3) programmes located in Europe, Australia or Canada; (4) an identifiable contact email. Ireland was included in the survey.

The final questionnaire (see online Supplemental Material) was developed using an online interactive GDPR-compatible survey tool (www.smartsurvey.co.uk). The questions were in English and most of the answers were in a pre-defined format; however free text format was also used, providing respondents with an opportunity to enter explanatory text and/or append supporting evidence of their policies/procedures. The online survey was subject to pre-testing and validations before going live. The questionnaire was sent via a link embedded in an emailed invitation to participate. This included a letter outlining the rationale for the research and what it would be used for. Respondents were advised they could contact the authors to clarify any points. Permission for anonymised publication was sought. A reminder mail was sent to country/programme representatives after 4 weeks to maximise response. Following receipt of questionnaires, discussions were held with some respondents to clarify queries arising.

Results

Seventeen of twenty-four countries/regions invited completed the survey, giving a response rate of 71%. Of the 17 countries/regions that completed the survey, 12 have an audit process in place for interval breast cancers that develop in the screened population (Table 1).

Table 1.

Key findings in those countries/regions undertaking audit.

	Country/region undertaking audit
Audit procedure	1	2	3	4	5	6	7	8	9	10	11	12
Audit of invasive breast cancers in the screened population	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Routine programme-wide review, with calculation of interval cancer detection rates	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓	✓
Routine individual patient cancer review	✓			✓	✓	✓
Only on patient/treating physician request
Routine sample of screened population
Other			✓		✓		✓
Radiology review of interval cancers	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Control images used	✓		✓		✓				✓
Radiology reviewers are blinded to cancer status of woman	✓				✓			✓
Audit procedure is different for cases requested for review by an individual patient	✓		✓				✓	✓	✓			✓
Patients are informed that a breast cancer audit is taking place		✓								✓
Women have a choice to be part of the audit					✓					✓
Capture consent from women to take part in a clinical audit	✓				✓		✓	✓			✓
At screening event	✓				✓		✓	✓			✓
Routine consent procedure covers the audit process	✓			✓	✓	✓	✓	✓	✓	✓	✓
Results of the clinical audit are communicated to the affected women	✓
Women are asked if they want to know the outcome of the audit	✓	✓								✓
Have an open disclosure policy for medical incidents	✓			✓	✓					✓		✓
Mandatory policy	✓			✓	✓					✓
Open disclosure extends to the results of audit of interval cancers in screening programme	✓									✓
Legal protection				✓	✓	✓		✓	✓
Financial compensation for interval cancers								✓				✓
Capture interval cancer rates for an internal report	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Publish interval cancer rates			✓	✓	✓		✓	✓				✓
Annual report			✓	✓	✓		✓	✓
Peer-review publication			✓				✓	✓				✓
On website							✓

Of the 12 who carry out audit, four carry out both a routine programme-wide review, with calculation of interval cancer detection rates, and routine individual patient cancer review. Seven countries/regions carry out a routine programme-wide review, with calculation of interval cancer detection rates only, while one country carries out audit on an individual basis and does not have a routine procedure in place currently (i.e., does not calculate programme ICRs or look at the total interval cancer burden, but facilitates review of prior mammograms at an individual woman's request). Ten of 12 respondents who undertake audit carry out radiological reviews.

Four countries/regions use controls (i.e., control women's mammograms) when carrying out a radiology review. Additional comments on the use of controls include the following: “Controls selected randomly by person coordinating review. Known to have 3 years normal follow up”; “For review the diagnostic mammograms and all available screening mammograms (last screening round and available screening rounds before) are reviewed”; “Random selection, but includes various breast density, review sets are enriched up to 30%”; “Reviews are run on a semi-informed scheme, i.e., the images from the last screening examination before diagnosis are shown in a series without information on the site where lesions subsequently developed”.

Three countries/regions carry out blinded radiology reviews, where the reviewers are unware of the woman's cancer status. In seven countries/regions the reviews are not blinded. Of the three countries/regions that carry out a blinded review, one country managed hindsight bias by having a case mix of interval cancer patients and patients with normal mammograms at subsequent screens while the other responded that three out of three blind reviewers must identify the cancer for it to be considered a false negative. The third country/region stated that two screening examinations before diagnosis are blindly reviewed by two experienced radiologists of the audit team. They have no prior knowledge of the clinical diagnostic mammograms, laterality or location of the interval cancer.

Of the seven who do not carry out a blinded review, three countries included details of their strategies to manage hindsight bias, which include the following: “Having a minimum of two people reviewing each case and subsequent regional review of all category two and three cases”; “Reviewers look at screening mammography first (unaware of the side and quadrant of interval cancer). Later they open also diagnostic mammography”; “Training”; “Only clearly visible lesions are called false negative.” The other two countries/regions commented “Acknowledge bias rather than managed” and “Perform audit for learning and quality improvement only”.

Of the 12 countries/regions that carry out audit, two countries/regions inform patients that an audit is taking place and two countries/regions give women a choice to be part of the audit. Nine countries/regions stated that their routine consent procedure for screening covers the audit process, so women would be aware that audit might take place but were not specifically informed when it was happening.

Of the five countries/regions that have an open disclosure policy for medical incidents, two stated that their open disclosure policy applies to interval cancers in screening. One other country/region has an open disclosure policy for category 3 interval cancer only (“suspicious”/false negative on review) as this is deemed a clinical incident. Three countries/regions ask women if they want to know the outcome of the audit.

Five countries/regions have legal protection in place for interval cancers arising in the screened population by means of the following: “Operating standard states that all interval cancers be reported to the state government's insurer”; “Proceedings of the blind review of interval cancers remain confidential under an Evidence Act, which is designed to facilitate open quality initiatives; “In the context of screening, the re-review procedure was introduced in 2018. Documents for interval cancers are scarce”; “Insurance for the health unit and the single specialist”.

One country/region has financial compensation for interval cancers, whereby this compensation depends on the decision of the insurance company, and it may or may not include a re-review organised by the expert centre for screening.

All 12 countries/regions capture ICRs for internal reporting; for example one country /region reports their rates to their national quality management committee. Six countries/regions publish their ICRs in their annual report. Four countries/regions also publish their ICRs in peer-reviewed publications and one country/region publishes their rates on their website.

Discussion

This is the first international comparative review of interval cancer audit processes in breast cancer screening in population screening programmes. We found from this survey that the management of interval cancer audit practices varies between international breast screening programmes. Five of 17 international programmes who responded do not examine interval cancers at any level. However, among programmes that do examine interval cancers, most focus on the calculation of the ICR, and some also conduct individual radiological reviews for quality improvement and educational purposes.

Huge benefits can be derived for screening when there is a fully functioning, robust, interval cancer audit and review process in place. The monitoring and review of interval cancers in screening is an integral part of quality assurance and programme improvement. In this survey most programmes use ICR as their key measurement for interval cancer quality assurance. The ICR can be benchmarked against national and international standards and provides reassurance that the programme is operating within appropriate quality parameters. The ICR is an internationally accepted key performance indicator (KPI) and is one of many KPIs used to assess ongoing programme performance. The ICR is calculated following cancer updates from a national or regional cancer registry. There is a normal and expected lag period between the index screening mammogram and when the interval cancers are notified to each programme. The ICR is therefore calculated a number of years after the index screening round occurred. Other surrogates of programme performance such as cancer detection rate, recall rate, positive predictive values, however, represent KPIs that are available on a more frequent basis and can reassure programme performance in a more timely fashion.

Individual interval cancer reviews represent an opportunity for radiologists to learn and improve their interpretative skills. Radiological review of interval cancers has been promoted as an educational tool by many. ⁹ Geertse et al. reviewed the routine audits of the Dutch Screening Programme and recommended that in addition to benchmarking screening outcomes, a radiological review of screening examinations and immediate feedback to the radiologist for education to improve interpretation should be part of an audit. ¹⁰ They concluded that radiological review provides insights into recall behaviour and cancer characteristics that cannot be gathered from epidemiological surveillance, and that case review of subtle changes facilitates improvement of radiologist skills. EUREF expects that breast screening programmes applying for accreditation will demonstrate that radiologists review individual interval cancer mammograms and in doing so will help improve radiology interpretative skills and future cancer detection; ³ however, there is no practice in breast screening that uses individual reviews as an audit parameter or as a measure of programme performance.

Individual reviews represent an opportunity for a patient or her physician to review the screening history and to see how the interval cancer might have developed. It is an opportunity to convey the limitations of screening and mammography. Communicating the limitations of screening, however, should not just follow when the interval cancer arises but prospectively from the first and every subsequent contact made with the programme; hence the importance of informed consent and good communications. A 2014 literature review of informed consent found that patients’ recollection and understanding of the medical procedure, risks and complications is often low.^11,12 For all screening programmes, education of the public regarding screening benefits and limitations is a key step in facilitating successful audit and disclosure.^9,12 An individual patient review can be an opportunity for a patient to review mammograms with the radiologist and help understand screening outcomes.

Individual systematic review and classification of interval cancers for audit purposes can be challenging because of the impact of hindsight bias. Although several programmes and authors offer mechanisms to limit hindsight bias by blinding the review process, drawing any conclusions from such retrospective classification processes and under such wide and varying conditions can be misleading. Hofvind et al. found that the proportion of missed/category 3 cancers varied widely depending on the review process used: lowest with a mixed blinded individual review (19.9%), higher in mixed blinded paired review (23.4%) and individual informed review (35.9%), and similarly high with consensus informed review (33.8%); 16.8% were reclassified as missed/category 3 when four or more radiologists assigned a score of 2 or more (probably benign or more suspicious) and 1.3% were reclassified as missed/category 3 when a score of 4 or more (probably malignant or more suspicious) was assigned. ¹³ Mullooly et al. found rates of false negative cancers varying from 4% to 40%; they also found variation in the number of interval cancers reviewed, in the blinding process, and in the classification of interval cancers used, with the number of categories varying from two to four or more. ¹⁴

While there is broad agreement that ICR and individual reviews are important components of healthy screening programmes, from this survey there is no consensus amongst many international programmes about how to conduct individual interval cancer reviews. Most programmes in practice do not have a standardised approach. Wide variation leads to big differences in the proportion of cases assigned to each category. The resulting difficulty with reproducibility is one of the reasons why sub-classification is not consistently utilised as a quality indicator in population screening programmes while the overall ICR is regarded as critical.

In this survey we ask about communication of audit results. Open disclosure policy in many countries refers to medical incidents where an unanticipated harm occurs to a patient. Only two countries confirmed that they contact and communicate audit findings to individual women. However, where there is instead a patient-requested review, there is immediate full disclosure of the findings.

The National Health Service Breast Screening Programme (NHSBSP) in England is the only programme that has published guideline documentation describing the open disclosure review processes that take place following individual case reviews for interval cancers. These processes were implemented in 2017 ¹⁵ and continue to evolve and adapt to the population screening environment in the UK. The NHS has estimated that 80% of interval cancers are new disease, 13% occult and seen only with hindsight bias, and 7% are missed cancers. ¹⁶ These proportions relate to a 3-year screening interval and may not directly relate to a 2-year interval. However, there is no other similar published figure for a 2-year interval programme.

A recent systematic review of open disclosure for breast screening found no study that conducted a retrospective radiological review purposely for open disclosure. ¹³ Retrospective reviews were conducted for audit/quality assurance and for research, including for radiologist education and learning. Variation in methodology was found across review type (non-blinded/semi-informed approach), number of reviewers and classification categories. The authors concluded that observed variation among radiological review practices likely impacts interval cancer classification results, and that reproducible and consistent methodology is required. Variation in the proportion of interval cancers classified as false negatives following radiological review reflects the variability in review methodology, classification systems employed and criteria for classification of each interval cancer category, variability in interpretation between individual radiologists, and the variable number of cases classified across studies. ¹⁷ When classifying interval cancers, it is also not unusual to find discrepancies across screening units, across screening programmes, and over time. ¹⁸

Most countries where population screening exists have not implemented an open disclosure for individual reviews. Interval cancers, even false negative cases, are in most screening programmes an expected and anticipated part of screening and are not considered a breach of clinical care or a significant clinical incident. If individual radiology reviews are to become part of interval cancer audit then screening programmes will require a significant change to the way interval cancers are regarded and reviewed. In addition to the challenges of agreeing and standardising any such review or audit process, each aspect would need to be agreed with all stakeholders and delivered consistently with ongoing review and validation. It may also require the introduction of a legislative system that would underpin population screening compared to diagnostic testing/activity. A clearer and more consistent approach to interval cancer reviews would also make clearer the link between classification and what action if any is warranted.

The response rate in this study was good and there was an opportunity to clarify responses directly if needed. However, we acknowledge that for those countries/regions where respondents may not have been aware of the Irish Breast Screening Programme, this was a “cold-call” email which may have deterred some from responding.

The findings of this survey along with other international consultation assisted the decision making of the established National Screening Service (NSS) ERG ⁸ , whose pertinent final recommendations are set out in Box 1. Since the ERG report was published, the NSS has worked with the National Cancer Registry of Ireland (NCRI) on a more detailed, GDPR-compliant protocol for the exchange of interval cancer information with specifically defined roles and responsibilities for the NSS and the NCRI. The programmatic ICR calculation should be a standard of quality assurance for all countries/regions doing breast screening. Defined roles and responsibilities are key to an accurate rate calculation.

Recommendation 1: Women should continue to be provided with all the information they require in order to help them make an informed choice to consent to participate in the BreastCheck programme. Current informational material should be revised in order to reinforce the information on the benefits and limitations of screening. This material should continue to include explicit information on the occurrence of interval cancers. It should also include information on how women can request a review of their case, if desired. Expanded content on data-sharing arrangements between BreastCheck and the National Cancer Registry of Ireland (NCRI) should be included.

Recommendation 2: BreastCheck should continue to monitor interval cancers at the programmatic level through the assessment of the interval cancer rate. The Expert Reference Group (ERG) recommends that the interval cancer rate should continue to be the main programmatic KPI used to monitor BreastCheck performance relating to interval cancers. Implementation of the recommendations of the Scally Report should ensure that communication with NCRI is strengthened to enable a more timely validation of interval cancers and the calculation of the interval cancer rate.

Recommendation 3: The ERG recognises the educational value of radiological review and classification of all interval cancers as recommended by EUREF. The ERG therefore recommends the development of technology which will allow blinded, anonymised radiological assessment of all interval cancers. In the absence of such technology, legislation that would facilitate this activity is recommended.

Recommendation 4: BreastCheck should continue to conduct patient-requested case reviews of interval cancers. The ERG further recommends that all patients diagnosed with breast cancer in Ireland should be asked if they have had a previous screening mammography performed. All interval cancer patients thus identified should then be offered a review of their previous screening mammography at a time which is appropriate to their care and after they have provided their informed written consent to BreastCheck. The consent should include a request to use the reviews for future educational exercises. The results of these reviews will be communicated to the interval cancer patient.

Recommendation 5: The findings of all patient-requested individual case reviews should continue to be disclosed using the BreastCheck standard operating procedure (SOP). The BreastCheck SOP is aligned with the current Health Service Executive (HSE) Open Disclosure Policy, and is consistent with the principles of open disclosure and professional ethical responsibilities.

Recommendation 6: The HSE should continue to build and promote understanding of, and public trust in, BreastCheck and other screening programmes through public information, engagement and education for participants, clinicians, and the wider society. Women should be made aware that they may, separately from any review process, request access to their imaging records at any time.

Box 1. Expert Reference Group recommendations – BreastCheck, the Irish Breast Screening Programme. ⁸

Individual reviews have clear benefits for patients and for radiologist learning; however, unless there is development of technology to allow blinded, anonymised radiological assessment of all interval cancers, this form of review and audit is unsustainable. The difficulty in reviewing and classifying interval cancers, the number of interval cancer cases each year, increasing legal exposure, and the ongoing belief that interval cancers are a failure of screening may have contributed to the small number of programmes conducting such standardised reviews. The Irish ERG recommends full anonymisation or legal protection for quality assurance if this form of review process is to be viable and radiologists are to be encouraged to participate (Box 1).

Conclusion

Programmatic measurement of ICRs provides quality assurance for breast screening programmes and allows for comparisons. In addition there are clear benefits from individual review for patients and for radiologist learning; however, unless there is development of technology that will allow blinded, anonymized radiological assessment of all interval cancers, as highlighted in the ERG recommendations, systematic review will continue to be a challenge.