10,589 CT pulmonary angiograms: evaluating the yield of acute pulmonary embolism

Abstract

Objectives:

To evaluate the change in the number of CT pulmonary angiograms (CTPAs) performed and the change in the yield of acute pulmonary embolism (PE) on CTPA at a busy tertiary teaching hospital from 2016 to 2019.

Methods:

All CTPA examinations for both in-patients and emergency department patients performed at our busy tertiary teaching hospital between 1 January 2016 and 31 December 2019 were identified from the radiology information system. A natural language processing technique called phrase matching was employed to assign each of the examination reports a result of either positive, negative or equivocal for acute PE. This algorithm was validated on a sample of 200 reports.

Results:

The number of CTPAs performed increased 59% from 2016 to 2019. The overall yield of acute PE has remained steady averaging 15.9%, ranging from 15.0% to 17.2%.

Conclusions:

Over 3 years, there has been a significant increase in the demand for CTPA examinations. The yield of acute PE has remained steady indicating a justified increase in demand. The yield of acute PE on CTPA within our centre is higher than the Royal College of Radiologists’ suggested minimum of 15.4% which suggests the current guidelines used for the investigation of suspected acute PE within our centre are appropriate.

Advances in knowledge:

The guidelines and subsequent yield of acute PE on CTPA at our tertiary teaching hospital can be used as a reference standard for other similar institutes.

Introduction

Getting It Right First Time (GIRFT) is a national programme of work aiming to improve medical care within the National Health Service (NHS) in England by reducing unwarranted variation. Taking a data-driven approach to tackling variation in the way services are delivered across the NHS and by sharing best practice, GIRFT identifies changes that will improve patient outcomes as well as delivering efficiencies and cost savings. Demand for clinical imaging is increasing year on year across England. ¹ This increase in demand is putting pressure on radiology departments to increase capacity, either through expansion or by improved efficiency. With this in mind, comparing the utilisation of clinical imaging across trusts in England was a specific focus of GIRFT. Through the work of GIRFT, our centre was identified as above the national average for the number of CT scans performed, ¹ this has led to several projects to investigate the usage of CT within our centre.

Our centre, a busy tertiary teaching hospital, is an adult patient only organisation with a major trauma centre, a national centre for pulmonary vascular disease as well as being a tertiary referral centre for many specialities including neurosurgery, orthopaedics and renal transplant. Over a 10-year period from 2009 to 2019, there has been an average 14% year on year increase in the number of CT pulmonary angiograms (CTPAs) performed at our organisation (Sheffield Teaching Hospitals NHS Foundation Trust). This study aims to investigate this increase in demand by assessing our performance and comparing our performance against recommended standards, to assess whether or not the increase in CTPA is meeting a justified demand.

The proportion of CTPAs on which acute pulmonary embolism (PE) is diagnosed (positive scan rate, PSR) is a commonly used metric to evaluate the usage of CTPA. PSR can be used to compare both the usage of CTPA between different institutes, as well as change over time within a single institute. Change in the PSR can be used to evaluate the impact of changes to guidelines and as an audit standard by comparing to national averages or agreed standards. If local guidelines are working and followed appropriately then the PSR should not be too low.

What constitutes an acceptable PSR is not widely established and may well vary depending on the patient population. The most widely cited values for an acceptable PSR have been published by The Royal College of Radiologists (RCR) who suggest an acceptable yield of acute PE on CTPA to be between 15.4 and 37.4%. ²

PSR is straightforward to calculate, the data can be extracted from a Radiology Information System (RIS) over a specified period of time and the reports analysed to assign them as positive or negative for acute PE. Within our centre, CTPA reports should include a clinical code at the end to denote whether the CTPA is positive, negative or equivocal for acute PE. These codes could be matched by software to calculate PSR efficiently for large numbers of patients.

PE is an important and frequent cause of morbidity and mortality. ³ PE presents a diagnostic challenge to clinicians because the signs and symptoms are vague and non-specific. ⁴ Therefore, clinicians need to maintain a high level of suspicion for PE in all patients presenting with dyspnoea, tachypnoea or pleuritic chest pain. ⁵ CTPA is the current golden standard test for the diagnosis of acute PE and a negative CTPA excludes PE. ⁶ The use of CTPA allows for rapid turnaround time whilst also evaluating for other potential causes of acute chest pain such as infection or rib fracture, diagnoses which may be occult on a chest radiograph. ⁷ There has also been a campaign with our organisation to raise awareness of venous thromboembolic disease, and to encourage clinicians to consider PE as a possible diagnosis. With these benefits in mind, and the drive to increase clinician awareness of venous thromboembolic disease, it is perhaps not a surprise that demand for CTPA has increased over the last 10 years.

The risks associated with CTPA are relatively low but there remains the theoretical risk of cancer due to ionising radiation as well as the risk of contrast induced nephropathy and the risk of adverse reaction including anaphylaxis to the intravenous contrast material. In view of these risks and to manage demand within the radiology department, CTPA should be reserved for patients where the probability of acute PE is relatively high. Pre-test probability tools along with D-Dimer testing can help clinicians with the challenge of selecting which patients require investigation with CTPA.

Our centre has guidelines for the investigation of suspected pulmonary embolism (Figure 1). These guidelines, adopted in 2016, are based on the 2012 National Institute for Clinical Excellence (NICE) guidelines, but with the addition of the pulmonary embolism rule out criteria (PERC). PERC is a pre-test probability tool for exclusion of PE in patients judged to be at low risk of acute PE in the emergency department.

Figure 1.

Flowchart demonstrating the protocol for the investigation of suspected acute PE in our institute. CTPA, CT pulmonary angiogram; CXR, chest radiograph; DVT, deep vein thrombosis; PERC, pulmonary embolism rule out criteria.

The two-level Well’s score is a well-established and validated clinical tool to estimate the pre-test probability of acute PE in patients, presenting as outpatients or to the emergency department, suspected of having an acute PE. ⁸ The two-level Well’s score uses the original Well’s score criteria (Table 1) to assign patients to either the low- or high-risk group. Those with a score of 4 or less are deemed low risk for PE and if a D-Dimer test is normal then the patient can safely be discharged with no further investigation or treatment, thereby avoiding the need for diagnostic imaging. Patients with a low risk Well’s score and a positive D-Dimer, or patients with a high risk Well’s score, require a CTPA to confirm or exclude the diagnosis of acute PE.

Table 1.

The Well’s score for pulmonary embolism

Clinical signs and symptoms of deep venous thrombosis (objectively measured leg swelling and pain with palpation in the deep-vein region).	3.0 points
Heart rate higher than 100 beats/min.	1.5 points
Immobilisation (bedrest, except to access the bathroom, for >= 3 consecutive days) or surgery in the previous 4 weeks.	1.5 points
Haemoptysis.	1.0 point
Malignancy (patients with cancer who were receiving treatment, those in whom treatment had been stopped within the past 6 months, or those who were receiving palliative care).	1.0 point
Pulmonary embolism as likely as or more likely than an alternative diagnosis.	3.0 points
Previous objectively diagnosed deep venous thrombosis or pulmonary embolism.	1.5 points

Table with the criteria and the score assigned to each one.

The PERC is another validated clinical tool which can be used for patients in the emergency department estimated to be at low-risk of acute PE. ^9,10 This would include patients with a Well’s score of 4 or less. ¹¹ Where the clinician estimates that the likelihood of PE is low, if none of the criteria (Table 2) are present then further investigation for PE is not required. A negative PERC, i.e. one which determines a low-risk patient to not require further investigation for suspected acute PE, is approximately 95% sensitive and has added benefits to the emergency department such as shorter attendances (median difference 37 min shorter, interquartile range 4 min to 1 h 8 min shorter) and cost saving of £30,002 to £20,426 (95% confidence interval) per 1000 patients. ¹⁰

Table 2.

The PERC, if any one of the criteria is met then further investigation for PE is required

Age ≥ 50. Heart rate ≥100. Oxygen saturation on room air of <95%. Unilateral leg swelling. Haemoptysis. Recent trauma or surgery. Prior pulmonary embolism or deep vein thrombosis. Hormone use (oral contraceptives, hormone replacement or oestrogenic hormones use in males or female patients).

PE, pulmonary embolism; PERC, pulmonary embolism rule out criteria.

If none are present then PE can be excluded without further investigation if the patient is judged at low risk by the clinician.

Where PE cannot be safely excluded through the use of pre-test probability tools and D-Dimer testing, CTPA is the imaging modality of choice. Ventilation-perfusion (V/Q) scanning is seldom used in our trust for the investigation of acute PE except in patients with an allergy to iodinated contrast due to the limited availability outside of normal working hours and location of the nuclear medicine department on a seperate site to our emergency department and acute medical and surgical units. CTPA remains the modality of choice for patients with renal dysfunction (estimated glomerular filtration rate, eGFR<30 ml/min/1.73 m²) after advice is sought from the renal medicine team about appropriate pre- and post-scan hydration and/or timing of renal replacement therapy.

In patients with shortness of breath or chest pain presenting to the emergency department, routine blood tests (including a clotting profile so that D-Dimer testing can be added on later without need for repeat phlebotomy) and a chest radiograph are requested by the triage nurse. These first-line investigations are then available for review when the patient is seen by an emergency department clinician. All patients require an up-to-date chest radiograph before requesting a CTPA. There is no cut-off for the grade/seniority of a doctor allowed to request CTPA in our centre but all requests have to be discussed with a radiologist. The chest radiograph is reviewed by the radiologist before vetting the CTPA request. If the chest radiograph suggests an alternative diagnosis, then further investigation with CTPA is not routinely indicated. It is important to note that there will be situations where an alternative diagnosis is suggested on a chest radiograph or blood tests but this is not concordant with the clinical picture and PE will remain suspected. Pulmonary infarction secondary to acute PE can look identical to pneumonia on a chest radiograph and congestive cardiac failure is an independent risk factor for acute PE. ^12–14 In these situations, if it would be appropriate to proceed with further investigation to exclude PE then we would typically ask for these requests to be made by experienced senior clinicians, usually a consultant.

Methods

All CTPA examinations performed for both in-patients and emergency department (ED) patients at our centre over a period of 3 years between 01 January 2016 and 31 December 2019 were extracted from the RIS database. These dates were chosen to include all patients since the publication of our centre’s current guidelines, whilst excluding patients imaged during the coronavirus pandemic from the beginning of 2020 onwards due to changes in the requesting of CTPAs during the pandemic. ¹⁵ Non-diagnostic studies were excluded.

The PSR for pregnant patients is significantly lower than non-pregnant patients, averaging 6.5% in one study by Goodacre et al. ¹⁶ Cost-effectiveness estimates in the Goodacre et al study suggest clinical features, decision rules and biomarkers do not accurately, effectively or cost-effectively select pregnant or post-partum females with a suspected PE for diagnostic imaging. Pregnant patients represent a small proportion (1%) of patients undergoing CTPA in our centre. ¹⁶ Patients undergoing assessment for pulmonary hypertension routinely have CTPA for exclusion of chronic pulmonary thromboembolic disease rather than acute pulmonary embolism. For these reasons, CTPA performed in pregnant patients and CTPA performed for assessment of pulmonary hypertension were excluded from the study.

The radiology report for the CTPA study performed in our centre has a diagnostic code to denote whether the result is positive, negative or equivocal for acute PE. These codes are added by the radiologist at the time of reporting the study. An algorithm was used to match these codes and assign the result as either positive, negative or equivocal. For reports which did not include a code, phrases used in the radiology report were matched from lists of phrases which identify a report as positive, negative or equivocal. For example, if the phrase “No acute pulmonary embolism.” were found in a report, the report would be deemed negative. If even after this no result could be assigned or matched the radiology report was manually reviewed and a result assigned. The algorithm to perform the phrase matching was written using the open source programming language python ¹⁷ utilising the openpyxl ¹⁸ module to read the data from an excel workbook containing the reports (see appendix for the code including the phrase lists). The list of phrases to match against was generated after review of CTPA reports from our centre within the year 2015.

A random sample of 200 reports were reviewed to evaluate whether a manually assigned result was concordant with the matched results using the algorithm.

Within our centre, requests for clinical imaging are made through an electronic system. When requesting a CTPA, clinicians are required to enter the Well’s score as well as which of the individual criteria from the Well’s score the patient meets. These data were extracted from the RIS. Examinations where there was discordance between the Well’s score as entered on the electronic requesting system and the Well’s score as calculated by the sum of the individual criteria entered on the electronic requesting system were excluded from any analysis involving grouping by the Well’s score because without review of the records for each of these patients individually, we could not be certain which of the totals (and therefore whether the patient was high or low risk) were correct.

Statistical analysis of the significance in differences in PSR between groups was made using Fisher’s exact test. For analysis of linearity of association between patient age and PSR, and change in PSR per year, χ² test for trend was used. Statistical analysis was performed using the SPSS Statistics software package. All averages are calculated as the mean unless otherwise stated.

Due to the retrospective nature of the study involving analysis of anonymised clinical data, informed patient consent and ethics committee approval was not required.

Results

11,097 CTPA requests were extracted from the RIS. 508 were excluded: 229 were pulmonary hypertension unit requests, 154 were incomplete scans and 125 were for pregnant patients. 10,589 CTPA requests were included in this study.

The algorithm to assign a result based on the radiology report was able to assign a result of either positive, negative or equivocal to 10,003 (94%) of the CTPAs. The remaining 586 reports were reviewed manually to assign a result.

From the sample of 200 reports reviewed to assess the validity of the algorithm, 18 had not been assigned a result by the algorithm as no matching phrases were found and 2 were not assigned a result as phrases were matched from both the positive and negative phrase lists. Of the 182 reports assigned a result by the algorithm, there was 100% concordance with manually assigned results. 128 (64%) of the reports contained a clinical code.

The overall positive scan rate (PSR) between 2016 and 2019 was 15.9% (Table 3). The PSR for inpatients over this period was 15.9 and 15.8% for ED patients with no significant difference between the two groups (p > 0.05). The overall PSR was greater than the RCR’s minimum target of 15.4% in each year from 2016 to 2018 but was 0.3% below in 2019 (Table 3), the 3 year average PSR was 0.5% above this target. There was no significant trend in PSR over the study period (p > 0.05).

Table 3.

Number of and percentage of CTPAs for in-patients and ED patients per year and their result for acute pulmonary embolism 2016–2019

		Equivocal CTPAs		Negative CTPAs		Positive CTPAs		Total CTPAs
Year	Patient type	Count	%	Count	%	Count	%	Count
2016	ED attender	51	7.5%	523	76.5%	110	16.1%	684
	In-patient	90	8.1%	827	74.8%	189	17.1%	1106
	Total	141	7.9%	1350	75.4%	299	16.7%	1790
2017	ED attender	63	5.8%	850	77.8%	179	16.4%	1092
	In-patient	163	8.6%	1430	75.5%	302	15.9%	1895
	Total	226	7.6%	2280	76.3%	481	16.1%	2987
2018	ED attender	47	5.1%	723	79.1%	144	15.8%	914
	In-patient	144	7.9%	1396	76.1%	294	16.0%	1834
	Total	191	7.0%	2119	77.1%	438	15.9%	2748
2019	ED attender	40	3.5%	936	81.5%	172	15.0%	1148
	In-patient	132	6.9%	1494	78.0%	290	15.1%	1916
	Total	172	5.6%	2430	79.3%	462	15.1%	3064
Total	ED attender	201	5.2%	3032	79.0%	605	15.8%	3838
	In-patient	529	7.8%	5147	76.2%	1075	15.9%	6751
	Grand total	730	6.9%	8179	77.2%	1680	15.9%	10,589

CTPA, CT pulmonary angiogram; ED, emergency department.

There was an increase in the number of CTPAs performed in 2019 compared to 2016, a 58 and 60% increase in the number of CTPAs for inpatients and ED patients respectively (Table 4), an extra 1264 CTPAs a year within the trust and an extra 164 PE diagnoses a year. In this same period from 2016 to 2019, ED attendances increased by only 5%, from 114,434 to 120,334. There was a significant increase (p < 0.0001) in the number of CTPAs performed per 1000 ED attendance from 15.6 in 2016 to 25.5 in 2019.

Table 4.

Table showing the PSR for patients in different age ranges

Age	ED patients PSR	In-patients PSR	Combined PSR
<35	8.5%	9.9%	9.2%
35–49	13.5%	12.8%	13.1%
50–64	16.9%	15.8%	16.2%
65–80	18.4%	15.9%	16.6%
>80	15.5%	18.2%	17.6%

ED, emergency department; PSR, positive scan rate.

ED patients, inpatients and then the overall PSR for each age range.

There was a significant increase in PSR with increasing age (Pearson’s χ² statistic = 17.19 (p < 0.01) and χ² for slope = 11.7 (p < 0.001)), from 9% in the under 30 s to 18% in the 70–80-year-old group (Table 4). The average age of the patient in each year ranged between 61 and 64 years across the study period.

74% of patients had a valid Well’s score. 5% of patients did not have a Well’s score recorded at all and 21% had a discordant Well’s score (the entered total score did not match the sum of the entered positive criteria).

There was no significant difference in the PSR between the Well’s high-risk patient group and the low-risk group (p > 0.05), 16.1 and 15.6% respectively (Table 5). Patients under the age of 35 with a high risk Well’s score, and hence proceeded straight to CTPA, had the lowest PSR of any patient group at 8.4% from 227 patients (Table 5). When looking into which criteria these patients were scoring on to achieve a Well’s score of more than 4, 98% scored 3 points for “PE likely”. Of these patients, 94% would have scored fewer than 4 points if they had not scored for “PE likely”.

Table 5.

Emergency department patients 2016–2019

	High-risk Well’s score (>4) group		Low-risk Well’s score (≤4) group
Patient age	PSR	Total number of CTPAs	PSR	Total number of CTPAs
<35	8.37%	227	10.19%	206
35–49	14.60%	322	10.92%	229
50–64	17.11%	380	17.32%	358
65–80	19.50%	441	19.78%	359
>80	18.13%	193	15.70%	121
Total	16.12%	1563	15.55%	1273

CTPA, CT pulmonary angiogram; PSR, positive scan rate.

PSR and total number of CTPAs for different age groups in the high- and low-risk Well’s score groups.

Discussion

NICE updated their guidelines for the investigation of venous thromboembolism (VTE) in March 2020. ¹⁹ The two major additions are the recommendations for use of age adjusted D-Dimer thresholds and to use PERC in low-risk patients. Our centre has been using PERC in ED since 2016.

Age-adjusted D-Dimer thresholds use an increased threshold for patients over the age of 50 where the threshold is equal to the patient’s age multiplied by 10. Age-adjusted D-dimer has now been included in our centre’s most recent guidelines for the investigation of suspected VTE, published in January 2022. A further study is being undertaken within our centre to combine D-Dimer results with the CTPA results obtained in this study to retrospectively evaluate the impact on the radiology department’s workload if age adjusted D-Dimer was introduced in our centre.

Maintaining standards with a PSR above a particular threshold is a marker that the diagnostic protocols and pathways in place are appropriate. There is not currently evidence to suggest how, if at all, changes in PSR affect patient outcomes. There is potential for morbidity and mortality if the diagnosis of acute PE is missed or delayed but this must be balanced with appropriate use of CTPA to manage the increasing demand for acute imaging. Achieving this balance is a shared responsibility between the clinicians and the radiologists. A collaborative approach to demand management, we believe is most likely to be successful and audit of PSR can aid this decision-making to balance patient outcomes with population radiation exposure, capacity within the hospital and cost effectiveness.

Our centre’s PSR between 2016 and 2019 is above the RCR’s minimum standard of 15.4%. The overall PSR of 15.4% is comparable to that reported by other UK centres, but there have not been any other centres who have published their PSR with more than 1000 CTPAs within the last 5 years. ^20,21 Current performance at our centre can serve as a benchmark for other similar centres.

There is one patient group identified in this study as having a PSR of less than 10%; patients under 35 with a high-risk Well’s score from ED. The PSR for this group was 8%. Looking at which Well’s criteria these patients were scoring on in order to reach the threshold over high risk, 94% of them were high risk due to “PE likely or more likely than an alternative diagnosis”. A PSR of only 8% makes PE relatively unlikely in this patient population in comparison to patients who are older or were deemed at low risk but subsequently had a positive D-Dimer. This raises the possibility that for this specific group of young patients, the “PE likely” criterion of the Well’s score is being overused at a population level. Education of the clinicians in ED as to the achieved PSR in different patient cohorts could help to improve the PSR in this patient group if there were a mismatch between the perceived prevalence by clinicians and the actual prevalence of PE. It is however important to note that this study does not include all patients with suspected acute PE in the low-risk group because there will be patients who have PE excluded on the basis of a normal D-Dimer or PERC and therefore would not have proceeded to CTPA and be included in this study. Furthermore, despite young, healthy patients having the highest long-term risk from radiation exposure, they also have the most benefit (measured in quality adjusted life years) from detecting and treating pulmonary embolism and therefore a lower PSR for young patients may be acceptable with the assumption that using clinical decision rules to exclude PE would result in missing a small proportion of PE. ¹⁶

21% of requests for CTPA from ED had a mismatch between the entered Well’s score and the marked criteria the patient was scoring on, this may be due to difficulty of use of the electronic system where requesters might not know how this should be used. There are prompts within the electronic requesting system to remind the user of the suspect PE protocol and if the entered Well’s score is 4 or less, then it will remind the requester that a D-Dimer should be requested first. Improvement to the ease of use of this electronic system would help to improve data quality for future audit. Other required inputs at the time of requesting of CTPAs include: a comment on the chest radiograph and the presence or absence of an alternative diagnosis, the present or absence of lower limb symptoms suggestive of DVT, the most recent eGFR, pregnancy status, patient mobility, infection control issues and presence or absence of contrast to iodinated contrast. Collation of this information into our RIS has many benefits; it ensures that this important information is available to the radiologist vetting the request, the radiographers in planning the scan and also for retrospective audit. For these reasons, we would recommend the use of clinical coding in other centres and expansion of their use across other imaging modalities and body parts.

Already, projects such as GIRFT are using a data-driven approach on a large scale to benchmark organisations. In the future, resource allocation decisions and changes to protocols at both a local and national level could be better informed by data produced by radiology departments. In our centre, as described already, diagnostic codes in reports and recording required inputs from the protocol in our radiology information system make this possible. The introduction of similar data collection processes in other hospitals and across other imaging modalities will be vital if we are to move towards standardised, protocol-driven pathways to improve efficiency within the radiology departments and patient outcomes both locally and nationally. We believe this is relevant to all hospitals, whether to benchmark against similar centres or to offer insight into performance of the same protocols across different patient populations and settings. A single national protocol for all centres may not be appropriate but without accurate data collection across a range of different organisations, this will be difficult to assess.

The algorithm used to automatically assign results had 100% concordance from 200 manually checked reports. This was deemed satisfactory performance and the algorithm was used to attempt to assign results for the remainder of the data. The use of algorithms like this, speeds up large-scale data collection for radiology audit and research. The algorithm’s phrase list was built for historical reports in a specific centre, its transferability to other centres could be limited due to different reporting styles or different clinical codes. The algorithm used phrase matching but more sophisticated natural language processing techniques could be employed to expand an algorithm’s utility, e.g. to identify alternative diagnoses identified by CTPA.

There are two limitations of this study, which although already discussed, should be emphasised. Firstly, this study only captures patients who have proceeded to CTPA for the exclusion of acute PE. Any patient who has undergone investigation for suspected acute PE but where PE has been excluded through application of clinical decision rules such as PERC or on the basis of a D-Dimer result as part of a two level Well’s score are not accounted for in this study. Whilst PSR is a useful parameter to monitor the performance of protocols and benchmarking against other centres in the context of CTPA, and a PSR within a certain range can be used to infer protocols are appropriate, PSR alone does not fully evaluate each step in the pathway for investigation of acute PE. Secondly, 25% of patients in this study were excluded from analysis of their Well’s score. This was due to incongruent recording of the Well’s score on the electronic request system or no Well’s score recorded at all. Whilst we don’t have reason to believe that there should be a correlation between a patient’s true Well’s score and those patients without a valid Well’s score, we have not further explored this due to difficulties associated with verifying the Well’s score from large numbers of hard copy clinical records.

Conclusion

Since the inception of 2016 guidelines for the investigation of acute PE at our institute, the achieved PSR from more than 10,000 CTPAs over a 4-year period has remained steady, averaging 15.9%, and has been above the RCR’s minimum standard of 15.4%. These findings suggest that we are meeting a justified demand with the use of CTPA for investigation of acute PE within our centre. Current performance and the protocols in use at our centre can serve as a benchmark for other similar centres.

The authors recommend the use of clinical codes and data collection within electronic requesting systems to allow efficient clinical audit across all image modalities but specifically in relation to the use of CTPA in the investigation of acute PE. These data could directly feed into national initiatives such as GIRFT to improve patient care and make efficiency gains through standardisation of clinical imaging pathways.