Establishing population-based surveillance of diagnostic timeliness using linked cancer registry and administrative data for patients with colorectal and lung cancer

Highlights

• •We developed a new methodology for measuring the secondary care diagnostic interval.
• •Secondary care diagnostic interval was calculated for 95% of cancers.
• •This interval varied by route to diagnosis, stage and comorbidities.
• •Routine GP referrals and outpatient routes had longest intervals.
• •Intervals decreased by advancing stage, within each route to diagnosis.

Abstract

Background

Diagnostic timeliness in cancer patients is important for clinical outcomes and patient satisfaction but, to-date, continuous monitoring of diagnostic intervals in nationwide incident cohorts has been impossible in England.

Methods

We developed a new methodology for measuring the secondary care diagnostic interval (SCDI - first relevant secondary care contact to diagnosis) using linked cancer registration and healthcare utilisation data. Using this method, we subsequently examined diagnostic timeliness in colorectal and lung cancer patients (2014–15) by socio-demographic characteristics, diagnostic route and stage at diagnosis.

Results

The approach assigned SCDIs to 94.4% of all incident colorectal cancer cases [median length (90th centile) of 25 (104) days] and 95.3% of lung cancer cases [36 (144) days]. Advanced stage patients had shorter intervals (median, colorectal: stage 1 vs 4 - 34 vs 19 days; lung stage 1&2 vs 3B&4 - 70 vs 27 days). Routinely referred patients had the longest (colorectal: 61, lung: 69 days) and emergency presenters the shortest intervals (colorectal: 3, lung: 14 days). Comorbidities and additional diagnostic tests were also associated with longer intervals.

Conclusion

This new method can enable repeatable nationwide measurement of cancer diagnostic timeliness in England and identifies actionable variation to inform early diagnosis interventions and target future research.

1. Introduction

Diagnosing cancer at early stages is associated with improved cancer survival [1,2] and better patient experience [3]. Regardless of stage, timely diagnosis is important, as tumours can progress whilst patients wait [4], survival declines with increasing tumour size even within the same TNM stage group [5] and shorter time intervals are usually associated with better outcomes for most cancers [6]. Furthermore, if too much time elapses during cancer diagnosis a patient’s level of fitness could deteriorate post-presentation, and the likelihood of them being fit to undergo radical treatment such as surgery reduces [7]. Faster diagnosis for later stage cancer patients may allow earlier symptom management and referral to palliative care or other support.

For many cancer patients, there is no routine monitoring of the length of the diagnostic pathway (and therefore speed of diagnosis) and any variation that exists allowing analysis or comparisons of change. Across the UK, devolved nations are assessing their diagnostic pathways and in England a proposed new metric will measure a particular cancer diagnostic interval in secondary care for specifically referred patients [8], increasing the importance of understanding these diagnostic pathways.

In England currently, the 62 day wait operational standard incorporates the diagnostic pathway, from an urgent GP referral for suspected cancer (Two Week Wait – TWW) or an urgent referral from the NHS Cancer Screening Programmes [9] to receiving first treatment. The operational standard of 85% of patients meeting the 62 day wait has not been achieved since 2014 [10]. Additionally, the 62 day wait only monitors referrals under TWW or screening routes, which in 2015 was 43% of all diagnosed cancers. 42% of colorectal and 28% of lung cancer patients are included in this target [11]. There is currently no routine measure of the pre-diagnostic intervals for all cancer patients, which would provide a more complete picture of different diagnostic pathway lengths.

Previous studies focusing on defined pathway time intervals [6,12] is plentiful, but increased availability of national level data allows examination of the entire secondary care diagnostic interval (SCDI) up to diagnosis. The Routes to Diagnosis (RtD) study has transformed our understanding of how patients are diagnosed [13] and identified initial presentation routes leading to a cancer diagnosis, these being Two Week Wait, Emergency Presentations, GP referral (routine), Other Outpatient, Inpatient elective, Screen detected, Death Certificate Only (DCO) and Unknown. RtD data has not been utilised previously to calculate time intervals of the diagnostic pathway, indeed the RtD algorithm identifies the date of an initial event for each patient (such as referrals to and outpatient hospital appointments) and provides the start of a patient’s secondary care pathway enabling calculation of a SCDI for all patients. Using linked Diagnostic Imaging Dataset (DID) data to cancer registrations provides another useful source [14] for identifying the first part of the diagnostic pathway in secondary care.

The aim of this work is therefore twofold: to devise a methodology to assign a SCDI for all cancer patients by determining the start and end of the diagnostic pathway in secondary care and secondly to examine variation and likely factors contributing to longer intervals for all patients diagnosed with two common cancers (colorectal and lung). This will identify patient groups most at risk of prolonged diagnostic intervals; inform the targeting of interventions or improvement efforts aimed at early diagnosis; drive further research; and monitor improvements.

2. Materials and methods

2.1. Data and exclusions

67 741 colorectal cancers (ICD10 codes: C18 – C20) and 74 904 lung cancers (C33 – C34) diagnosed in English residents in 2014 and 2015 were identified using the cancer registration data held by the National Cancer Registration and Analysis Service (NCRAS), Public Health England (PHE) and formed our two cohorts of interest. Patients were excluded (Fig. 1) if they were diagnosed via their death certificate, if they had multiple tumours of the same site since 2012 or if, from the RtD study, their diagnostic route was unknown [13] due to lack of events prior to diagnosis. Following clinical advice, lung cancer patients with an unclear stage 3 (i.e. a stage 3 coded as something other than 3A or 3B) were also excluded.

Fig. 1.

Flow diagram to show the exclusions applied to the cohorts.

Ethnicity was self-reported and submitted to the cancer registry from hospital patient administration systems records. Deprivation quintile was assigned using the income domain from the 2015 Index of Multiple Deprivation [15]. The Charlson score was used for comorbidities identified in the timeframe in the cancer registration and hospital records, combining the score of other defined conditions (e.g. diabetes, pulmonary disease, cardiovascular disease), using methods previously described [16] with a different time window between 27 and 3 months prior to cancer diagnosis [17]. Stage was categorised into individual TNM stage groups 1, 2, 3 and 4 for colorectal cancer, whilst for lung cancer patients, stage was regrouped into 1&2, 3A and 3B&4 to better represent shared characteristics and treatment for early- and late- stage lung cancers.

2.2. Data linkage

Our study builds on the RtD study linkage [13] to include further diagnostic events and timings in order to calculate the SCDI. The cancer registrations were linked at patient level to the following routine health datasets, which provided dates and other information about specific events taking place during diagnostic pathways:

• -Cancer Waiting Times (CWT) [18] – used to assign urgent referrals for suspected cancer (TWW) - referral to CWT, first seen in secondary care
• -Routes to Diagnosis (RtD) [13] – diagnostic route – referral to a hospital outpatient appointment or hospital attendance
• -Hospital Episode Statistics (HES) [19] – diagnostic procedures - including colonoscopy and bronchoscopy
• -Diagnostic Imaging Dataset (DID) [20] – diagnostic imaging – including chest X-Rays (CXR) and chest Computed Tomography (CT) scans

2.3. Determining the SCDI

In order to measure the SCDI, it is appropriate to follow a similar method to the RtD study [13] and work back from the cancer diagnosis record in order to determine the most likely start point of the diagnostic pathway.

2.4. End of SCDI: diagnosis date

The end point of the diagnostic pathway, and of the SCDI, is a cancer diagnosis, achieved for all on the cancer registry. Diagnosis dates in cancer registries are determined by the European Network of Cancer Registries (ENCR) rules, where a hierarchy of definitions of diagnosis is used, the highest being the date of pathological verification [21]. This may be justified as the basis for robust and internationally comparable statistics but is unhelpful for purposes of measuring diagnostic intervals. This is because, in practice, for some patients the treating clinicians would reach a firm diagnosis of cancer (e.g. as indicated by initiation of anti-cancer therapy) before the date of diagnosis according to ENCR rules. There is also a significant minority of patients in whom there is no pathological confirmation of a clinical diagnosis, in lung cancers diagnosed in 2015 this was over a quarter of patients [22]. We therefore considered the date of occurrence of a Multi-Disciplinary Team (MDT) meeting, or recorded treatment referral or treatment initiation, as defining the diagnosis date if they occurred within the 28-day period before the date of diagnosis defined by ENCR rules (Fig. 2). If one of these events occurred more than 28 days before the diagnosis date, we retained the registry diagnosis date, as the derived date was likely to relate to a different tumour. This approach resulted in re-assigning of the diagnosis date held in the cancer registry in 11% and 10% of colorectal and lung patients, respectively.

Fig. 2.

Calculation of the SCDI.

2.5. Start of SCDI: definition of first event

A start of the SCDI is also required in order calculate the interval. For patients referred to an urgent pathway for suspected cancer (TWW) and those diagnosed via screening there is a clear starting point of this interval, well captured and monitored by NHS England as part of CWT dataset [17]. For cancer registrations diagnosed via other diagnostic routes there is no recorded start of this interval, so we established a first event for every patient using the possible events in the diagnostic pathway in the linked data within 6 months before diagnosis (Fig. 2). For each site, potential first events in the diagnostic pathway were defined in consultation with clinicians, including verification of relevant imaging and procedure codes and likely secondary care interactions or events expected prior to colorectal and lung cancer diagnosis (Appendix – Table A.0). The date of the first event (e.g. HES outpatient appointment, DID chest X-Ray) occurring in the six months before diagnosis was retained for each patient (Fig. 2). This time period chosen was determined with clinical advice. Other diagnostic tests, in addition to the first event, were also investigated using DID and HES in relation to the diagnostic pathway for both sites, these were CT Colonography for colorectal cancer patients, and Positron Emission Tomography (PET), Single Photon Emission Computerised Tomography (SPECT), Bronchoscopy and Endobronchial Ultrasound (EBUS) for lung cancer patients.

2.6. Statistical analysis

Descriptive statistics present the SCDI (calculated by subtraction between SCDI end/start) by socio-demographic variable (age group, sex, deprivation quintile and ethnicity), disease (stage, comorbidities) and other factors (diagnostic route, other diagnostic events). The SCDI was categorised into a ‘long/short’ length for statistical analysis, where ‘long’ was an interval greater than the median for that cancer site.

Two univariate logistic regression models investigated the associations between individual covariates and a long SCDI, for colorectal and lung patients, respectively. Covariates were added into multivariate logistic regression models. In the lung cancer model the number of chest imaging procedures was also added. The covariates in the final model were determined by univariate analysis, previous literature and advice from clinical colleagues. Likelihood ratio tests were carried out to determine the overall significance of the associations of each variable with the outcome variable. Analysis was carried out using STATA v15 [23].

3. Results

A SCDI was calculated for 63 958 colorectal and 71 361 lung cancer patients. The most common first events identified as the start of this interval for colorectal cancers were referral onto an urgent suspected cancer pathway CWT & RtD referral on the same date (28%), referral to a hospital outpatient appointment (RtD referral: 19%), CWT referral (16%), hospital attendance (RtD start date: 15%) and colonoscopy (7%). For lung cancer first events were a relevant image (CXR or chest CT: 61%) or a relevant image along with a CWT or RtD event on the same day (17%). There were 362 (0.5%) colorectal and 165 (0.2%) lung cancer patients where no event was found in the 6 months prior to diagnosis and these were therefore excluded from the analysis (Fig. 1).

The median (Inter-Quartile Range – IQR; 90th centile) interval was 25 days (10–52; 104) and 36 days (15–85; 144) for colorectal and lung cancer respectively. Table 1, Table 2 present SCDIs for patient-tumour characteristics alongside regression analysis results for colorectal and lung cancer respectively. The interval varied significantly for both sites by stage and diagnostic route. For both cancer sites, patients diagnosed at early stage had longer median intervals than those diagnosed at late stage (34 vs 19 days for stage 1 vs 4 colorectal cancers; 70 vs 27 days for stage 1&2 vs 3B&4 lung cancers). This pattern was observed overall and within each diagnostic route (Figs. 3 & 4 ).

Table 1.

Secondary care diagnostic interval (SCDI) & regression results: Colorectal cancer.

	n (%)	SCDI (days) centiles				Unadjusted	Adjusted
		25th	50th	75th	90th	OR (95% CI)	OR (95% CI)
Total cohort	63 958 (100)	10	25	52	104
Age group							p < 0.0001
<25	327 (0.5)	0	1	3	43	0.14 (0.10-0.19)	0.35 (0.24-0.52)
25-44	2 035 (3.2)	2	18	55	104	0.76 (0.69-0.84)	0.92 (0.81-1.04)
45-49	1 524 (2.4)	8	24	55.5	104	0.96 (0.86-1.08)	0.95 (0.83-1.09)
50-54	2 869 (4.5)	10	24	52	104	0.95 (0.87-1.04)	0.95 (0.86-1.05)
55-59	4 261 (6.7)	11	25	55	105	0.99 (0.91-1.07)	0.95 (0.87-1.04)
60-64	6 562 (10.3)	14	25	48	98	Reference
65-69	8 575 (13.4)	13	25	51	103	0.99 (0.93-1.05)	0.96 (0.90-1.04)
70-74	9 428 (14.7)	14	26	54	106	1.08 (1.02-1.15)	1.03 (0.96-1.11)
75-79	9 808 (15.3)	12	27	58	112	1.11 (1.04-1.18)	1.05 (0.98-1.13)
80-84	9 279 (14.5)	10	27	55	104	1.09 (1.03-1.17)	1.08 (1.00-1.16)
≥85	9 290 (14.5)	4	20	47	96	0.75 (0.70-0.80)	0.92 (0.85-0.99)
Sex							p = 0.0002
Male	35 318 (55.2)	11	25	52	103	Reference
Female	28 640 (44.8)	9	25	53	105	0.99 (0.96-1.02)	1.07 (1.03-1.11)
Ethnicity							p < 0.0001
White	57 851 (90.4)	11	25	53	104	Reference
Black	861 (1.4)	10	27	56	106	1.17 (1.03-1.34)	1.38 (1.18-1.62)
Asian	1 151 (1.8)	10	26	61	116	1.07 (0.95-1.20)	0.99 (0.86-1.13)
Mixed & other	9 879 (1.5)	10	25	50	108	1.03 (0.91-1.17)	1.09 (0.94-1.27)
Not known	3 116 (4.9)	7	21	42	89	0.77 (0.71-0.83)	0.85 (0.78-0.93)
Deprivation quintile							p=0.0444
1 - least deprived	14 251 (22.3)	11	25	53	104	Reference
2	14 517 (22.7)	12	25	53	104	1.02 (0.97-1.07)	1.06 (1.00-1.11)
3	13 496 (21.1)	11	25	53	105	1.00 (0.95-1.05)	1.05 (0.99-1.11)
4	11 999 (18.8)	9	24	51	101	0.97 (0.92-1.01)	1.09 (1.03-1.15)
5 - most deprived	9 596 (15.6)	8	24	52	105	0.93 (0.88-0.98)	1.07 (1.00-1.13)
Stage							p < 0.0001
1	10 184 (15.9)	18	34	75	126	Reference
2	15 212 (23.8)	13	27	55	105	0.67 (0.64-0.70)	0.81 (0.77-0.86)
3	17 189 (26.9)	13	25	51	100	0.61 (0.58-0.64)	0.72 (0.68-0.76)
4	14 892 (23.3)	5	19	37	80	0.38 (0.36-0.40)	0.56 (0.53-0.59)
unknown/other	6 481 (10.1)	3	19	50	108	0.44 (0.41-0.47)	0.68 (0.62-0.73)
Diagnostic route							p < 0.0001
Emergency Presentation	15 469 (24.2)	1	3	12	37	0.17 (0.16-0.18)	0.18 (0.17-0.19)
GP Referral (routine)	15 183 (23.7)	31	61	102	143	4.31 (4.11-4.53)	4.43 (4.21-4.65)
Two Week Wait	20 755 (32.4)	17	25	38	61	Reference
Inpatient Elective	2 113 (3.3)	0	6	27	57	0.37 (0.33-0.41)	0.37 (0.34-0.41)
Other Outpatient	4 092 (6.4)	18	49	96	145	2.23 (2.08-2.40)	2.19 (2.03-2.35)
Screen detected	6 346 (9.9)	18	27	45	92	1.14 (1.08-1.21)	1.14 (1.07-1.21)
Charlson comorbidity score							p < 0.0001
0	50 548 (79.0)	10	24	48	95	Reference
1	6 395 (10.0)	11	28	72	128	1.30 (1.24-1.37)	1.31 (1.24-1.40)
2	3 660 (5.7)	11	29	74	133	1.33 (1.24-1.42)	1.37 (1.27-1.49)
3+	3 355 (5.3)	8	27	70	128	1.16 (1.08-1.24)	1.33 (1.22-1.45)
Additional diagnostics*							p < 0.0001
No	59 394 (92.9)	9	23	49	101	Reference
Yes	4 564 (7.1)	30	50	86	126	4.98 (4.62-5.38)	4.58 (4.21-4.98)

Bold text – statistically significant associations.

^*Colorectal cancer additional tests in 6 months before diagnosis - CT Colonography.

Table 2.

Secondary care diagnostic interval (SCDI) & regression results: Lung cancer.

	n (%)	SCDI (days) centiles				Unadjusted	Adjusted
		25th	50th	75th	90th	OR (95% CI)	OR (95% CI)
Total cohort	71 361	15	36	85	144
Age group							p < 0.0001
<25	35 (0.1)	8	24	75	123	0.79 (0.40-1.54)	0.67 (0.29-1.54)
25-44	720 (1.0)	12	29.5	72	134.5	0.81 (0.69-0.94)	0.86 (0.71-1.03)
45-49	1 104 (1.6)	13	30	66	126	0.77 (0.68-0.88)	0.90 (0.77-1.04)
50-54	2 354 (3.3)	14	31	73	133	0.81 (0.74-0.89)	0.91 (0.82-1.02)
55-59	4 279 (6.0)	16	34	76	136	0.92 (0.85-0.99)	0.98 (0.90-1.07)
60-64	7 335 (10.3)	17	35	79	139	Reference
65-69	11 449 (16.0)	17	36	84	141	1.02 (0.96-1.08)	1.01 (0.94-1.08)
70-74	12 445 (17.4)	18	40	88	147	1.16 (1.09-1.22)	1.15 (1.07-1.23)
75-79	12 355 (17.3)	16	41	92	150	1.18 (1.11-1.25)	1.17 (1.09-1.25)
80-84	10 055 (14.1)	14	38	93	149	1.11 (1.05-1.18)	1.18 (1.10-1.27)
≥85	9 230 (12.9)	7	28	81	141	0.82 (0.77-0.88)	1.10 (1.02-1.19)
Sex							p = 0.9836
Male	38 149 (53.5)	15	36	85	144	Reference
Female	33 212 (46.5)	14	36	85	144	0.98 (0.95-1.01)	1.00 (0.97-1.04)
Ethnicity							p = 0.0002
White	65 763 (92.2)	15	37	86	145	Reference
Black	648 (0.9)	13	33	86	147	0.86 (0.74-1.01)	0.94 (0.78-1.13)
Asian	984 (1.4)	16	42	99	153	1.15 (1.01-1.31)	1.04 (0.89-1.21)
Mixed & other	891 (1.3)	13	31	75	133	0.81 (0.71-0.92)	0.90 (0.77-1.06)
Not known	3 075 (4.3)	9	25	61	128	0.60 (0.56-0.65)	0.82 (0.75-0.89)
Deprivation quintile							p=0.2647
1 - least deprived	10 172 (14.3)	16	36	85	143	Reference
2	12 796 (17.9)	16	36	84	146	0.99 (0.94-1.05)	1.02 (0.96-1.08)
3	14 039 (19.7)	15	36	85	143	0.98 (0.94-1.04)	1.04 (0.97-1.10)
4	15 823 (22.2)	15	36	85	144	1.00 (0.95-1.05)	1.06 (1.00-1.13)
5 - most deprived	18 531 (26.0)	14	36	86	146	0.97 (0.92-1.02)	1.02 (0.96-1.08)
Stage							p < 0.0001
1&2	16 521 (23.2)	34	70	123	162	Reference
3A	8 021 (11.2)	22	42	88	143	0.48 (0.45-0.50)	0.53 (0.50-0.57)
3B&4	41 506 (58.2)	11	27	61	124	0.25 (0.24-0.26)	0.38 (0.37-0.40)
unknown/other	5 313 (7.5)	8	38	104	155	0.39 (0.37-0.42)	0.70 (0.65-0.75)
Diagnostic route							p < 0.0001
Emergency Presentation	24 906 (34.9)	4	14	52	126	0.48 (0.46-0.50)	0.36 (0.34-0.37)
GP Referral (routine)	16 657 (23.3)	32	69	126	163	2.59 (2.48-2.71)	2.56 (2.43-2.69)
Two Week Wait	20 415 (28.6)	22	35	63	107	Reference
Inpatient Elective	1 240 (1.7)	9	18	43	107	0.43 (0.37-0.48)	0.37 (0.32-0.43)
Other Outpatient	8 143 (11.4)	30	66	122	161	2.38 (2.25-2.51)	2.00 (1.88-2.12)
Charlson comorbidity score							p < 0.0001
0	49 079 (68.8)	14	31	67	121	Reference
1	10 110 (14.2)	20	53	116	161	1.91 (1.83-2.00)	1.75 (1.66-1.84)
2	5 982 (8.4)	22	63	126	162	2.25 (2.12-2.38)	2.09 (1.95-2.23)
3+	6 190 (8.7)	22	75	139	169	2.51 (2.37-2.65)	2.46 (2.30-2.62)
CXR and CT count							p < 0.0001
0	4 766 (6.7)	5	15	41	96	0.60 (0.56-0.65)	0.55 (0.50-0.60)
1	10 540 (14.8)	7	25	58	114	Reference
2	27 295 (38.3)	12	26	52	110	0.92 (0.88-0.96)	0.98 (0.93-1.03)
3+	28 760 (40.3)	31	67	123	161	3.84 (3.66-4.02)	5.25 (4.97-5.54)
Additional diagnostics*							P < 0.0001
No	60 848 (85.3)	13	32	79	141	Reference
Yes	10 513 (14.7)	34	61	108	155	2.99 (2.85-3.12)	2.12 (2.01-2.24)

^*Lung cancer additional tests in 6 months before diagnosis- PET, SPECT, Bronchoscopy, EBUS.

Fig. 3.

Secondary care diagnostic interval (SCDI) by diagnostic route and stage, Colorectal cancer.

Fig. 4.

Secondary care diagnostic interval (SCDI) by diagnostic route and stage, Lung cancer.

By diagnostic route, the longest intervals were seen for those who were diagnosed via routine GP referrals (colorectal cancer median 61 days; lung 69 days), followed by outpatients (colorectal cancer 49 days; lung 66 days). The shortest intervals were seen in patients who were diagnosed after an emergency presentation (3 days for colorectal cancers; 14 for lung), though, in the emergency route, 10% of patients had intervals longer than 37 and 126 days for colorectal and lung cancer respectively.

Patients with comorbidities had longer intervals, as did those with additional diagnostic tests, where median intervals exceeded 50 days for both sites.

There was strong evidence of association between diagnostic route and interval after adjusting for other factors. Compared with patients diagnosed via TWW, those diagnosed via an emergency had lower odds of a long interval (OR 0.18 (95%CI [0.17–0.19]) for colorectal cancer, 0.36 [0.34–0.37] for lung). Patients diagnosed through routine GP referrals had significantly higher odds of a long interval (4.43 [4.21–4.65] for colorectal cancer, 2.56 [2.43–2.69] for lung), as well as patients diagnosed via the outpatient route (2.19 [2.03–2.35] for colorectal cancer and 2.00 [1.88–2.12] for lung). For both sites there was a decreased likelihood of having a longer interval with advancing stage, where stage 4 cancers had the lowest odds of a long interval (colorectal cancer 0.56 [0.53–0.59]) compared with stage 1 and 0.38 [0.37–0.40] for stage 3B&4 for lung compared with stage 1&2.

For both sites, patients with a Charlson comorbidity score of three or more had increased odds of having a long SCDI compared with patients with no comorbidities (colorectal 1.33 [1.22–1.45], lung 2.46 [2.30–2.62]). Additional diagnostic procedures for colorectal and lung cancer patients resulted in increased odds of a longer interval and lung cancer patients having more than two chest x-rays or CT scans before diagnosis had an OR of 5.25 [4.97–5.54] compared with only one CXR or CT. The risk of having a long interval increased as patients got older, peaking in 80–84 year olds for colorectal cancer (1.08 [1.00–1.16]) and lung (1.18 [1.10–1.27]), before decreasing in the oldest age groups.

Female colorectal cancer patients had a higher odds of having a longer interval (1.07 [1.03–1.11]), where no such pattern was found in lung cancer patients. There was also an increased risk of having a longer interval for colorectal cancer patients in more deprived quintiles, with ORs of 1.09 [1.03–1.15] and 1.07 [1.00–1.13] for the two most deprived quintiles compared with the least deprived, where no significant associations were found for lung cancer.

4. Discussion

A secondary care diagnostic interval (SCDI) has been established for the first time for nearly all patients belonging to the English incident cohort of colorectal and lung cancer patients, using robust decision-rules to calculate the interval from first relevant presentation/diagnostic test to diagnosis. The method is scalable to other cancer sites and repeatable over time enabling continuous monitoring of diagnostic timeliness nationwide.

The findings indicate that a large proportion of colorectal and lung cancer patients experienced SCDIs exceeding 30 days. Early stage, routine GP and outpatient diagnostic routes, greater number of comorbidities and additional diagnostic investigations were all associated with longer SCDIs. Shorter SCDIs for patients presenting via emergency or TWW pathways were evident at all stages of disease, and with advancing stage. For colorectal cancer, females and the more deprived had increased odds of having longer SCDIs, not evident in lung cancer.

Our work expands on the RtD study, incorporating time to event-based data used to define diagnostic route in order to establish diagnostic length [13]. While our work does not map exactly onto other intervals which have been reported in the literature, such as the secondary care interval [11] or time from referral to secondary care to diagnosis [6], it does provide a measurement of a key part of the diagnostic process for many cancer patients, the SCDI, incorporating tests ordered in primary care which occur in a secondary care setting.

Studies using primary care data (from either electronic health records dataset or as part of a national audit initiative) have reported relatively longer intervals than those reported in our study [24,25]. This could reflect that by their nature these studies were able to use intervals with time points starting in primary care (unlike our method which measures intervals from when the patient is first seen in secondary care, even if for a test requested in primary care). Other prior research has examined patient interval (symptom onset to presentation to primary care) and primary care interval variation [26,27], but that evidence is by necessity limited to pre-referral aspects of the diagnostic process and is often sourced through bespoke data collection or non-national sources.

Other conceptually similar attempts have been made in the US using SEER-Medicare (claims) data [28], and in Denmark [29], using data from a well-characterised sub-regional cohort but the approach used in our study has the benefit of being more easily scalable and repeatable, as it is based on linkage of national cancer registration and routine healthcare utilisation data. This makes it the most comprehensive study of SCDIs to date. Use of rich health datasets linked to established cancer registrations achieves a more complete picture of diagnostic events of cancer patients in secondary care, irrespective of route to diagnosis and importantly includes the routine referral route, via which significant proportions of cancer patients are diagnosed. An important limitation is that our interval begins at diagnostics or secondary care interactions, due to the lack of primary care data coverage for England, and so the entire cancer diagnosis pathway is not captured. Additionally data linkage issues probably accounted for the lack of first event, and therefore exclusion from SCDI calculations, for a very small proportion of our cohorts.

Longer SCDIs were evident for lung cancer, explained partly by diagnostics (CXR particularly) which can be requested in primary care rather than the secondary care-based colorectal cancer diagnostics. Between cancer sites comparisons are difficult, however, due to differences in both diagnostic procedures and symptom development [26]. The TWW pathway is by its design intended to confer a sense of urgency, but the results suggest that the imperative for a patient to be seen by a specialist within two weeks does not ensure a swift pathway and diagnosis thereafter, with a quarter of colorectal and lung cancer patients diagnosed via TWW having intervals longer than 38 and 63 days respectively. This may reflect post-assessment waiting for specialist tests due to capacity constraints [30], suboptimal test selection, incomplete or inadequate tests (which may need to be repeated), or alternative tests. Older and less fit patients may also experience longer than average intervals when preparation is needed for invasive investigations such as colonoscopy.

It is well recognised in the literature that shorter diagnostic intervals are not always associated with better outcomes [6] and shorter intervals for emergency presentations or later stage disease are likely for patients with more pronounced symptoms, who are less likely to have positive outcomes due to the severity of their disease [31,32]; the sicker-quicker effect. Patients who have experienced extended intervals but have early stage disease may reflect less aggressive disease or could be in line with an incidental finding of cancer, where symptoms experienced may not have actually been caused by the cancer but were sufficient to prompt investigation during which cancer was detected. Nevertheless, for patients other than those presenting with advanced stage disease, shortening of SCDIs can help improve outcomes, including survival [6] and mortality [33]. It may also lower the risk of disease progression and can help improve their experience of the health service.

There are many benefits in using routine data surveillance of these phenomena, allowing scalability, ability to monitor progress over time and across patient groups. The same methodology can be applied to other cancer sites, providing appropriate diagnostic tests are known, to guide improvements in diagnostic care and capacity allocation. Repetition with more recent cancer data would enable changes since the implementation of the National Institute for Health and Care Excellence (NICE) suspected cancer recognition and referral guidelines in 2015 [34] to be assessed, specifically any impact on SCDIs. Further refinement of the approach could include measurement between different points along the diagnostic pathway, to further explore pinch points and opportunities for service optimisation.

5. Conclusion

We address a substantial knowledge gap for earlier cancer diagnosis by developing an approach to measure SCDIs in national incident cohorts of cancer patients. In patients with colorectal or lung cancer, significant interval variation exists by various patient and disease factors. Our baseline measures of these intervals can help with understanding of complex disease, service, tumour and patient factors associated with longer intervals. Knowledge of SCDIs offers a baseline assessment and a useful indication of the degree of improvement needed, and where focus is required, in order to ensure patient diagnostic pathways are as rapid as possible to optimise patient experience and potentially improve outcomes.

Author’s contributions

Conception and design of the work: CP, JF, JM, GL and JS

Linking, cleaning, checking and analysis of the data: CP, JF, SM

Drafting of manuscript: CP with significant input from JF, JS and GL.

Clinical advice on study design and definitions: RV (colorectal) & MP (lung)

All authors made substantial contributions to the interpretation of the findings

All authors contributed to revising the manuscript critically for important intellectual content and approved the final version

All authors have agreed to be accountable for all aspects of the work.

Funding

GL is supported by a Cancer Research UK Advanced Clinician Scientist Fellowship (award C18081/A18180). GL is an associate director (co-investigator) of the multi-institutional CanTest Research Collaborative funded by a Cancer Research UK Population Research Catalyst award (C8640/A23385)

Conflict of interest

The authors declare no conflict of interest

Appendix A. Supplementary data

The following is Supplementary data to this article: