One step closer to quantifying ‘clinical likelihood’ in pre-test probability

This editorial refers to ‘Prediction models as gatekeepers for diagnostic testing in angina patients with suspected chronic coronary syndrome’, by L. Bjerking et al. doi:10.1093/ehjqcco/qcac025.

In patients presenting with stable chest pain, pre-test probability is a key tenet driving investigation and management decision across guidelines. This is true whether this be the European Society of Cardiology (ESC),¹ the American societies,² or the UK National Institute of Clinical Excellence (NICE) guidelines.³ In patients with a low pre-test probability, no further work-up is required, while in those with an intermediate or high pre-test probability (PTP), further work-up is recommended. Thus the prognostic ability of PTP models for a given population has important implications for both patients and healthcare systems.

Despite the centrality of the concept of PTP, all of these guidelines differ in the models they recommend for the assessment of this. The PTP model recommended by NICE only considers the typicality of chest pain, recommending further investigation in typical or atypical chest pain, or non-cardiac chest pain with electrocardiogram (ECG) changes.³ The rational for this was the poor performance of the Diamond Forrester (DF) risk score in contemporary populations for the prediction of obstructive coronary artery disease.⁴ To address this issue the most recent ESC and American guidelines recommend the use of an updated DF risk score (henceforth known as the ESC PTP model for simplicity).⁵ This was derived from a meta-analysis of multinational data incorporating the PROMISE trial,⁶ the CONFIRM registry,⁴ and a Danish registry,⁷ to provide evidence from a combined population of 15 815 patients (mean age of 59 ± 11 years old, 52% female).⁵ The key strength of this updated PTP calculation is its derivation in a large cohort of patients from 14 countries encompassing Europe, America and Asia. While this significantly improved the discrimination of the ESC PTP score, a recent study examining this in the PROMISE trial showed that physician judgement of the probability of coronary artery disease was more accurate than the ESC PTP for the prediction of obstructive coronary artery disease and risk of cardiovascular events.⁸ In this regard the ESC guidelines were prescient in their inclusion of a modifying step—the physician determined ‘clinical probability’ which should be used when the PTP is 5–15%. While this is a key concept in the guidelines, this ‘clinical probability’ assessment is relatively undefined, with risk factors listed, but with no guidance for how a clinician should weight these to help guide downstream management. A recent study in the Eastern Denmark Heart registry sought to define and weight some of these risk factors, providing a PTP calculation including hypertension, dyslipidaemia, diabetes mellitus, and a family history of coronary artery disease.⁷ This improved the risk discrimination compared with a model including only age, sex, and nature of chest pain, with the C-statistics improving from 0.86 (95% CI 0.84–0.88) to 0.88 (95% CI 0.86–0.90, P < 0.0001). While the model showed good calibration within this derivation cohort, no external validation was performed.

It is this question of external validity that Bjerking and colleagues seek to address in this issue of the Journal.⁹ They assess the previously derived clinical model on an external population from the Western Denmark Heart Registry, including 42 328 patients (57.3 ± 11.3 years old, 54% women) who underwent coronary computed tomography angiography (CCTA) between 2008 and 2017. In this population they compare the ESC PTP calculator with the previously developed clinical model incorporating age, sex, type of chest pain, hypertension, dyslipidaemia, diabetes mellitus, and a family history of coronary artery disease for the prediction of obstructive coronary artery disease. Importantly, the authors find good risk discrimination of both the ESC PTP, and the clinical model with no significant differences between these (AUC for the ESC PTP was 0.76 [0.75–0.77] and 0.76 [0.75–0.76] for the clinical model (P = 0.77 for differences)). This provides a robust external validation of the ESC PTP calculator, supporting its use in clinical practice.

However, from a clinical perspective, perhaps the more important PTP thresholds are 5% (no further investigation necessary) and 5–15% (clinical probability to be considered). They are likely to have a greater impact on patient management than the stratification of intermediate and high risk, as these lower thresholds determine whether any further investigation is required. It is in these low risk strata that the new clinical model appears to perform best, whereas the ESC PTP over-estimates the risk of coronary artery disease. The new clinical model estimated lower risk in the majority of patients, classifying 55.7% of patients as being at <5% risk, compared with 19.5% from the ESC PTP model. This was correct in 96.5% of cases with no obstructive disease being present, and 3.5% obstructive disease prevalence (1.9% higher than the 1.6% obstructive disease prevalence in the low risk group of the ESC PTP). The healthcare implications of this reclassification are massive. In this Danish cohort of just over 42 000 patients, almost 15 000 more patients could avoid further cardiac imaging and investigation based on this PTP model. When considering those with a PTP < 15% the differences reduced but were still substantial with 83% considered to be at <15% compared with the clinical model vs. 64.5% using the ESC PTP model, at a cost of missing 1.9% more cases with obstructive coronary artery disease.

This study does suffer from some limitations. The study only examined those who underwent CCTA. This preferentially selects a low risk cohort, missing those referred direct to invasive coronary angiography or for functional testing, with this reflected in the low prevalence of coronary artery disease in the current cohort. The gold standard used for the identification of obstructive coronary artery disease was the presence of a flow limiting lesion on pressure wire assessment, or a 50% stenosis on invasive coronary angiography. This meant that the true prevalence of obstructive disease in this cohort will have been underestimated as there will be a cohort of patients in whom CCTA showed a >50% stenosis in whom medical therapy was chosen with no subsequent invasive investigation undertaken. In addition, the cohort represented patients from one region in Denmark, which may not be representative of patients in other countries.

So what are the clinical implications of these findings? For the people of Denmark, this study represents an excellent step forward, providing a more structured and quantitative approach to the integration of clinical probability into pre-test probability. It allows for the safe deferral of a much larger number of patients from further investigation. However generalisation and transportation of the studies findings into other populations must be viewed with caution. The prevalence of coronary artery disease in the current cohort is much lower than that in the populations used in the development of the ESC PTP model (8% in the current cohort vs. 15% in the ESC cohort). Further, while no ethnographic information was available in the current study, previous studies in Denmark suggest that this is likely to be ∼98% Caucasian, reducing its applicability to more diverse populations.¹⁰ Previous work from the SCORE2 consortium showed the susceptibility of cardiovascular risk prediction models to both the prevalence of disease and risk factors.¹¹ Thus while the current study shows the benefit of including clinical variables into the calculation of PTP, the weighting of these will require recalibration before use in other countries.

A final question remains regarding how best to manage patients in the recalculated 5–15% range of PTP. One option might simply be to treat any underlying risk factors, and reassure the patient as to the low probability of coronary artery disease, and the even lower probability of cardiovascular events. A recent study of the ESC PTP model using data from the SCOT-HEART trial showed both the very low risk (PTP < 5%) and low risk (PTP 5–15%) cohorts to have a very low rate of cardiovascular events after 5 years of follow up, (1.4% and 1.5% respectively), with no difference demonstrated between these groups.¹² Only those with a PTP > 15% had an elevated risk of cardiovascular events (4.1% at 5 years). Not all doctors or patients will necessarily be satisfied with such a hands-off approach. One solution might be coronary artery calcium scoring, which both the ESC and ACC/AHA guidelines support for lower risk patients in whom further assessment is deemed to be of utility. Several recent studies have shown calcium score alone to be superior to PTP calculations even in chest pain patients across several international cohorts.^13,14 It is also cheaper, quicker, and more accessible than CCTA. Even this approach is not without its limitations. A CACS of 0 is still associated with a 2–8% risk of obstructive coronary artery disease in those presenting with chest pain,¹⁵ with this ranging from 3% in those under 40 years old to 8% in those over 80.¹⁶

The best solution will of course remain that which both the patient and treating physician are comfortable with. This conversation will undeniably be aided by calculated and perceived risk, likelihood of events, and comfort with starting preventative medications. The work of Bjerking and colleagues does an excellent job of making this a more robust and informed conversation, and will hopefully spur on similar efforts in other healthcare settings.

Funding

JRWM is supported by the NIHR Cambridge Biomedical Research Centre (BRC-1215-20014). MCW (FS/ICRF/20/26002) is supported by the British Heart Foundation. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Conflict of interest: none declared.