Testicular cancer: improving outcomes with national quality performance indicators

Abstract

Testicular cancer is the most common malignancy in young adult men. The prognosis is excellent in limited disease and cure is possible even in advanced disease. Quality performance indicators (QPI) are used in many developed countries as a measure of healthcare performance. We report and discuss the development of a national set of QPIs in Scotland for testicular cancer as a method of gathering demographic data and driving improvement in nationwide testicular cancer outcomes.

Introduction

Testicular germ cell tumours are relatively rare. In 2016, a total of 232 new cases were diagnosed in Scotland with a crude incidence of 8.1 cases per 100 000 of the male population, making it the 16th most common cancer in men in Scotland. It takes on a greater significance than numbers alone might suggest as it is one of the few curable solid cancers even when it has metastasised, with a crude overall 5-year survival rate in Scotland of 98.7%.¹

Delays in diagnosis affect the stage of disease at presentation and therefore the prognosis.^{2 3} Minimising further delays from cancer diagnosis to treatment also influences outcome, resulting in improved survival and quality of life.⁴

Historical papers suggested that quality of care differed depending on the treatment centre.⁵ Specifically, in testis cancer in Scotland, a trend to worse outcomes was demonstrated in lower volume hospitals.⁶ This led to increased centralisation of services, with treatment at specialised cancer centres⁷ and a recommendation for this process in the recent European Society of Medical Oncology guidelines.⁸

In Scotland, the model of care for patients with a diagnosis of testicular cancer may differ depending on the geographical site of diagnosis. When patients present to hospitals that fall within the catchment areas of tertiary cancer centres, that is, hospitals in Glasgow, Edinburgh, Dundee, Aberdeen and Inverness, they are automatically discussed at a regional multidisciplinary team (MDT) meeting. However, if a patient presents to a district general hospital, they may follow the above referral pathway or they may be discussed at a local MDT before referral onwards to the regional MDT. The regional cancer centres can therefore have multiple feeder hospitals (table 1).

Table 1.

Current regional distribution of NHS health boards in Scotland

Region	Regional MDT site	Health boards
WoSCAN	Beatson West of Scotland Cancer Centre, Glasgow	NHS Greater Glasgow and Clyde NHS Ayrshire and Arran NHS Lanarkshire NHS Forth Valley
SCAN	Western General Infirmary, Edinburgh	NHS Lothian NHS Dumfries and Galloway NHS Borders NHS Fife
NoSCAN	Ninewells Hospital, Dundee Aberdeen Royal Infirmary, Aberdeen Raigmore Hospital, Inverness	NHS Tayside NHS Grampian NHS Orkney NHS Highland NHS Shetland NHS Western Isles

MDT, multidisciplinary team; NHS, National Health Service; NoSCAN, North of Scotland Cancer Network; SCAN, South-East Scotland Cancer Network; WoSCAN, West of Scotland Cancer Network.

To maximise healthcare outcomes, quality performance indicators (QPI) were developed and introduced. They act as a proxy for quality of care. These are measurable, agreed standards and enable uniform data to be collected across a country. They are used to monitor areas of healthcare performance such as effectiveness, efficiency, safety and quality.^{9 10} This is important as it allows identification of areas that are performing well and also those areas that need improvement. QPIs also address the variation in quality of cancer services, which is pivotal to delivering improvements in quality of care.

Lack of a formal national framework for the management of cancer or a formal set of standards makes collecting national data on specific tumour sites and subsequently improving healthcare performance difficult. The QPI process allows for cross-region comparison and discussion at a national level at a clinician-initiated annual meeting.

In this article we report our experience in the development of a national set of QPIs in testicular cancer, which have enabled us to collate national testicular cancer data, identify deficits in our care framework and attempt to implement interventions to optimise performance and improve patient outcomes. This was made possible by comprehensive data collection from a population of approximately 5.4 million people over a several year period.¹¹ We will give examples of where we were performing well and where improvement was needed. We will also discuss the challenges around gathering a large data set and the potential for using it for future projects.

Methods

QPI development

The first step in the process was to form a QPI development group. The membership of this group consisted of clinical representatives from each of the regional cancer networks in Scotland (including oncologists, urologists, radiologists and pathologists) along with patient/carer and government representation. It was called the Testicular Cancer QPI Development Group and was established in November 2013. There was a breadth of experience within the group but the chairperson was required to have had prior experience in the QPI process.

In order to determine what potential QPIs should be included, a systematic search was carried out by Healthcare Improvement Scotland (HIS) using selected websites and two primary medical databases to identify national and international guidelines. The scope for development of testicular cancer QPIs and a search narrative were defined and agreed by the development group.

Twenty-one guidelines were appraised for quality using the Appraisal of Guidelines for Research and Evaluation II instrument.¹² This instrument assesses the methodological rigour used when developing a guideline. Fifteen of the guidelines were not recommended for use with the remaining six used. Table 2 shows the final search criteria used in the literature search.

Table 2.

Search criteria used to identify guidelines to be used to identify QPIs¹⁸

Inclusion	Exclusion
Primary testicular cancer including: Seminomas Non-seminomatous germ cell tumours Germinomas Teratocarcinomas Diagnosis staging Surgical management of disease Non-surgical management of disease (chemotherapy, radiotherapy) Surveillance	Related cancers, including: Lymphomas Leydig and Sertoli cell tumours Recurrent disease/relapsed disease management Primary care/referral Precancerous conditions including carcinoma in situ/testicular intraepithelial neoplasia (TIN) Prevention Screening Clinical trial recruitment and protocols. Symptom management (eg, nausea and vomiting, neutropenic sepsis) Communication, information sharing and support Palliative/end-of-life care (pain management, end-of-life counselling, hospice management)
Age range: adults Date: 2005 to present day Language: English only Document types: clinical guidelines

QPI, quality performance indicator.

Using the identified guidelines, the development group set out to create evidence-based, measurable indicators with a clear focus on what could make a real difference to quality of care (figure 1). Subgroups were formed (diagnosis, staging and surgical treatment and non-surgical treatment) and identified draft QPIs that fulfilled the following three criteria: were important to quality and outcome, were evidence based and were measurable.

Figure 1.

Quality performance indicator (QPI) development process.¹⁸ ISD, Information Services Division.

Once QPIs were identified, an engagement period followed whereby the draft QPIs were widely circulated. Clinical and management colleagues from across National Health Service (NHS) Scotland, patients affected by testicular cancer and the wider public were given the opportunity to provide feedback. Feedback received was considered by the group and appropriate adjustments made where required. If differences in opinion arose, they were discussed, taking into account the available evidence, and a group consensus decision reached.

The QPIs were designed to be clear and measurable, based on sound clinical evidence while also taking into account other recognised standards and guidelines. Each QPI was given a short title to be used in reports. A fuller description was also written, which explained exactly what the indicator was measuring, with specific criteria for inclusion and exclusion, followed by an overview of the evidence base and rationale, which explained why the development of this indicator was important.

The next step was detailing measurability specifications, highlighting how the indicator would be measured in practice to allow for comparison across NHS Scotland. Finally, a target was indicated, suggesting the level which each unit should be aiming to achieve against each indicator. Flexibility in these targets was allowed due to inherent confounding factors.

An example of this is demonstrated in QPI 1—radiological staging. CT scanning is an essential part of the staging of all germ cell tumours. We know that timely imaging is important to ensure timely treatment decision-making and that unnecessary delays can have an impact on prognostic groups and hence survival rates. The time point of 3 weeks from orchidectomy was decided based on the fact that due to postoperative wound healing, chemotherapy is unlikely to be commenced within 4 weeks of surgery. Ideally, the target would be 100% but tolerance is built into the target to allow for patient choice, for example, they may not be able to attend their allocated appointment. A more lenient tolerance is facilitated in the surveillance imaging described in QPI 9, again reflecting patient factors, allowing for unavailability, lack of compliance with follow-up and those who relapse prior to their scans.

Table 3 shows the agreed QPIs and their targets. Of note, QPIs 10 and 11, which relate to mortality and clinical trials, are standard QPIs for all tumour types. The generic nature of QPI 10 in particular has a limitation as we would want tumour site-specific targets.

Table 3.

The final testicular cancer QPIs v2.0 (21 June 2016)

Testicular quality performance indicators	Target (%)
1. Radiological staging—proportion of patients who undergo CT scanning, ideally contrast-enhanced CT, of the chest, abdomen and pelvis within 3 weeks of orchidectomy.	95
2. Preoperative assessment—proportion of patients who undergo preoperative assessment of the testicle which includes (1) serum tumour markers (STM) and (2) testicular ultrasound.	95
3. Primary orchidectomy—proportion of patients who undergo primary orchidectomy within 2 weeks of ultrasonographic diagnosis. (Hospital of surgery)	95
4. Multidisciplinary team meeting—proportion of patients discussed at an MDT meeting to agree a definitive management plan after orchidectomy.	95
5. Pathology reporting—proportion of patients undergoing orchidectomy where the pathology report contains full information (as per the Royal College of Pathologists (RCoP) data set).19	90
6. Quality of adjuvant treatment—proportion of patients with stage I seminoma receiving adjuvant single-dose carboplatin within 8 weeks of orchidectomy.	95
7. Serum tumour markers—proportion of patients with metastatic testicular cancer who undergo STMs 2 weeks before starting chemotherapy.	98
8. Systemic therapy—proportion of patients with metastatic testicular cancer who undergo systemic anticancer therapy (SACT) within 3 weeks of an MDT decision to treat with SACT.	95
9. CT scanning for surveillance patients—proportion of patients with ‘low-risk’ testicular non-seminomatous germ cell tumour (NSGCT) (or mixed) under surveillance who undergo at least three CT scans of the abdomen±chest and pelvis within 14 months of diagnosis. (Note: year 1 reporting)	85
10a. 30-day mortality (orchidectomy)—proportion of patients who die within 30 days of treatment for testicular cancer. (Hospital of surgery)	<5
10b. 30-day mortality (chemotherapy)—proportion of patients who die within 30 days of treatment for testicular cancer.	<5
10c. 30-day mortality (radiotherapy)—proportion of patients who die within 30 days of treatment for testicular cancer.	<5
11. Clinical trial—all patients should be considered for participation in available clinical trials, wherever eligible.	>15

MDT, multidisciplinary team; QPI, quality performance indicator.

Implementation of QPIs

The Scottish Government’s Better Cancer Care (2008)¹³ included an obligation to develop indicators of quality. The Scottish Cancer Task Force introduced the National Cancer Quality Steering Group tasked with the development of generic and tumour-specific QPI sets and a national governance framework, which mandated NHS Scotland’s territorial health boards, to comply with this process.¹⁴ After the QPIs were finalised, they were circulated to individual health boards. Each NHS health board nominated a QPI clinical lead who was responsible for ensuring all members of the MDT were aware of the QPIs and relevant targets. Pathways for clinical care, from diagnosis to treatment, were reviewed at a local level and suggestions for improvement were discussed and implemented where relevant.

Data analysis

Each new diagnosis of testicular cancer is registered at a local level. The NHS board is then required to track the progress of each individual patient through the cancer pathway to produce a minimum core data set according to QPIs. The NHS boards collate and validate the data on a yearly basis and an annual regional report is produced by the regional networks. This allows areas of best practice and variance to be highlighted.

Improving survival forms an integral part of the national cancer quality improvement programme. To ensure consistent application of survival analysis at specific time points, it was agreed that a single analyst on behalf of all three regional cancer networks would undertake this work. The annual regional report is therefore submitted to the central Information Services Division (ISD) for collation and publication.

Mechanism for quality improvement

Following central analysis of the QPI data, an expert group of clinicians, with HIS assistance, review comparative national results and write to regional centres highlighting areas of good practice and variance. In cases of variance, NHS boards may be requested to submit improvement plans. HIS is available to provide support and expertise on improvement methodologies if required. Progress is monitored and if not acceptable, HIS will visit the service concerned and work at a local level to address the issues. Following this visit, a report will be submitted to the Scottish Cancer Task Force with an improvement proposal to take forward to the Scottish Government Health Department.

Initial pilot and QPI review

As part of the process of the refinement of the QPIs, they were initially piloted for 1 year. Following baseline review and publication of results at the end of this period, some changes were made to measurability in an attempt to ensure that the QPIs appropriately measured what they were intended to. The review process was undertaken by senior clinicians and in consultation with various other clinical specialties.

After this review, it was agreed that formal review of all QPIs should take place following 3 years of national reporting with revisions being implemented from year 4 onwards (currently under way). At each review the overall importance of the indicator along with the most recent high-quality clinical evidence and the measurability will be considered. If required, the QPIs will then be adjusted according to the need.

Results

Overall

We present a single prospectively comprehensive set of national data, with figures collected and collated from the three geographical regions—North of Scotland Cancer Network, South-East Scotland Cancer Network (SCAN) and West of Scotland Cancer Network (WoSCAN) (table 1). As described above, the data are collated in each region and then forwarded to a national centre. The data also enable us to look at individual hospital site’s performance if required. As part of this process, each regional clinical lead has access to the data and is able to comment on areas of poor performance prior to national reports. This encourages active feedback locally and promotes quality improvement development.

It is important to note that when analysing the regional and national data, the cancer audit is a dynamic process with patient data continually being revised and updated as more information becomes available. This means that apparently comparable reports for the same time period and cancer site may produce slightly different figures if extracted at different times. Moreover, due to small numbers in certain centres, there will be background and regional variation often resulting in small increases and decreases in performance. All of these factors are taken into account when the report is produced. The key focus of the QPI system is to identify consistently large variations that cannot be explained by confounding factors.

Patients may be excluded from individual QPIs in certain situations (tables 3 and 4). We present data from the first three yearly reports, obtained between October 2014 and September 2017. Over this time period there were 525 recorded diagnoses of testicular cancer in Scotland: 139 in 2014/2015; 213 in 2015/2016; and 173 in 2016/2017 (table 5). The results for the last 3 years of national data are presented in table 6.

Table 4.

QPIs and potential reasons for exclusion

Quality performance indicators	Potential reason for exclusion
1. Radiological staging	Patients undergoing chemotherapy prior to orchidectomy.
2. Preoperative assessment	Patients who refuse to undergo assessment. Patients undergoing chemotherapy prior to orchidectomy.
3. Primary orchidectomy	Patients undergoing chemotherapy prior to orchidectomy.
4. Multidisciplinary team meeting	No exclusion.
5. Pathology reporting	No exclusion.
6. Quality of adjuvant treatment	Patients who are treated within a clinical trial.
7. Serum tumour markers	No exclusion.
8. Systemic therapy	Patients whose primary chemotherapy management is as part of a chemotherapy clinical trial.
9. CT scanning for surveillance patients	Patients who have received adjuvant chemotherapy. Patients who are treated within a clinical trial.
10. 30-day mortality	No exclusions.

QPI, quality performance indicator.

Table 5.

Number of cases of testicular cancer in Scotland over time

	Overall
	2014/2015	2015/2016	2016/2017
Cases, n
Total	143	212	164
Seminoma	85	128	101
NSGCT	58	84	63

Available information regarding pathology. Some figures differ from the article text due to the dynamic nature of reporting.

NSGCT, non-seminomatous germ cell tumour.

Table 6.

The QPIs identified and implemented and the corresponding national targets and results

Definition	Target	Scotland
		2014/2015	2015/2016	2016/2017
QPI 1—radiological staging	95%	88%	84%	89%
QPI 2—preoperative assessment	95%	98%	94%	92%
QPI 3—primary orchidectomy	95%	51%	49%	53%
QPI 4—multidisciplinary team (MDT) meeting	95%	99%	99%	97%
QPI 5—pathology reporting	90%	98%	99%	99%
QPI 6—quality of adjuvant treatment	95%	73%	81%	86%
QPI 7—serum tumour markers	98%	100%	86%	97%
QPI 8—systemic therapy	95%	88%	76%	89%
QPI 9—CT surveillance scanning for patients	85%	N/A	96%	80%
QPI 10a—30-day mortality (orchidectomy)	<5%	0%	0%	0%
QPI 10a—30-day mortality (chemotherapy)	<5%	0%	3%	0%
QPI 10a—30-day mortality (radiotherapy)	<5%	0%	0%	0%

Green=target met. Red=target not met.

N/A, not applicable; QPI, quality performance indicator.

Example of good performance

From table 6 we can see that we have maintained a very low national 30-day mortality rate. We have also demonstrated consistently high national standards in timescales for pathology reporting and multidisciplinary meetings. Of note, the five patients who were not discussed in MDT prior to orchidectomy in 2016/2017 had valid reasons including requirement of urgent treatment of their disease.

Examples of poor performance and subsequent intervention

Urban areas with large catchment populations and underdeveloped referral pathways to multiple hospitals provide particular challenges for delivering services and meeting targets due to demands. This is highlighted by one particular urban area which only achieved 16/34 (47%) of orchidectomies being performed within 2 weeks. This is compared with the target of 95%.

Following feedback of this result to the regional board, a solution was proposed in the form of a clinical nurse specialist (CNS) one-stop clinic, whereby direct referral could occur and patients, in a single visit, could be seen by a CNS±urologist, receive an ultrasound, have tumour markers performed and a CT scan requested (if required), with surgery planned by the end of the visit. The patient would then be listed for the regional MDT at an appropriate time.

This intervention is in the process of being established and it is hoped it will improve performance. A similar scheme has been introduced in Birmingham, UK, and has been shown to produce a median time from clinic to having an orchidectomy of 5 days.¹⁵

A further example can be seen in the QPI based on the quality of adjuvant chemotherapy for patients with seminoma. Looking at this QPI in more detail, it was noted that regions that performed better tended to have centralised treatment, supporting the argument for this model of care. On a regional level, poor performance was noted in WoSCAN in 2014/2015 and 2015/2016. When this was reported, root cause analysis identified failings in the referral pathway as a contributing factor. These have since been addressed with a subsequent improvement in performance.

Trials

The QPI definition of an interventional trial is ‘a clinical study in which participants are assigned to receive one or more interventions (or no intervention) so that researchers can evaluate the effects of the interventions on biomedical or health-related outcomes. The assignments are determined by the study protocol. Participants may receive diagnostic, therapeutic, or other types of interventions’. A translational study is defined as ‘a term used to describe the process by which the results of research done in the laboratory are used to develop new ways to diagnose and treat disease.’

The data set demonstrated that our recruitment to clinical trials, both interventional and translational, was poor.

Discussion

Testicular cancer is a rare and curable disease. Centralised treatment is known to improve outcomes. We have shown that it is possible to collect data on a unique nationwide level and use it to report national cancer outcomes in a specific tumour site by the development of site-specific QPIs. The results enable active feedback to regions with the hope of driving improvement at a local level. Within Scotland there are several tumour-specific patient databases that this model could be applied to.

A similar model has been applied successfully on a regional level in Belgium¹⁶ and we believe we are the first country to show the application of QPIs to a specific tumour site on a national level.

The establishment of national QPIs required a period of consultation involving multiple healthcare professionals from various backgrounds, as well as patient and carer input. Once standards had been agreed, data were collected and annual performance reports produced.

The results obtained enabled performance to be viewed on local, regional and national levels; however, interpretation of performance at a local level can be challenging due to the small numbers of cases. The annual reports enable us to see the areas that we are performing well in but also identify areas that need improvement and potential reasons why.

It is important to emphasise that the QPIs and the relevant targets are under constant review. An example is the modification to the orchidectomy QPI ahead of year 4. Due to pressures on service provision nationally, the significant challenge to meeting a 2-week target from diagnosis to surgery was recognised. Following review of the clinical evidence it was felt that the time frame could be extended to 3 weeks without adversely affecting patient outcomes.

In addition, following on from the example of Birmingham’s one-stop clinic,¹⁵ a similar scheme is being established in Glasgow, while NHS Fife has implemented a new ‘hot-list’ clinic and SCAN and NHS Forth Valley have implemented improved referral pathways. These changes were a direct response to the QPI data with the aim of enabling faster diagnosis and treatment as well as theoretically reducing the risk of complications and helping alleviate patient anxiety. These interventions promote intraregional and inter-regional learning, further driving national quality improvement.

As intimated above, our results highlight the stresses placed on the UK’s NHS and the implications of lack of recruitment to certain specialties, for example, radiology. In our population, we have struggled to meet the required standard for obtaining CT staging and surveillance CT scans within the required time frame. This has added importance as the results can affect treatment decisions and outcomes. In response to the year 2 QPI data, SCAN introduced a new process throughout the region to request CT scans prior to surgery, which resulted in a significant improvement in year 3—indeed they were the only network to meet the 95% target. This again highlights the changes that can be driven by QPI data.

As well as the individual QPI results, the creation of such large patient databases enables the collection of large quantities of reliable data across various parameters which could provide the potential for exploratory analysis, for example, the role of socioeconomic status in testicular cancer outcomes.

Despite the advantages of national QPI data, we do however recognise potential limitations in the accuracy of data collection. With any large data collection, accurate coding is a necessity, which is subject to human error. In addition, faults in computer algorithms may potentially result in errors and a lack of data. Anecdotally, errors were seen in WoSCAN where it was initially recorded that no patients had been recruited to clinical trials, yet on investigation the trial network confirmed this was not the case.

In an attempt to minimise such occurrences, ISD data quality assurance is regularly performed.¹⁷ In this process, a sample of records from each board across Scotland is taken and checked for accuracy. Although only a snapshot, despite variance between boards, the overall accuracy was 95.7%, which is considered to be very good. A report is produced every time this is done and recommendations are given which aim to help improve consistency of recording and interpretation.

It should also be noted that in groups with small numbers such as in QPI 9, the decision of a patient, not unreasonably, to delay the start of chemotherapy for low-volume metastatic disease until after family events such as a wedding can lead to a significant drop in annual performance figures from an area. It is therefore more meaningful to look at performance over a number of years.

Overall, the QPI process and results generated have driven change and enabled identification of system errors within local, regional and national frameworks. Our hope is that the ongoing review of performance will continue to improve healthcare performance and ultimately patient outcomes on a national level.

Conclusion

We have shown that it is possible to develop a set of tumour-specific QPIs for this relatively rare cancer and to subsequently collect a comprehensive set of patient data at a national level. We have also shown it is possible to use these data to identify areas for improvement in service provision and patient care. The data that are collected can potentially be used for future observational studies.