The Manchester Acute Coronary Syndromes (MACS) decision rule for suspected cardiac chest pain: derivation and external validation

Abstract

Objective

We aimed to derive and validate a clinical decision rule (CDR) for suspected cardiac chest pain in the emergency department (ED). Incorporating information available at the time of first presentation, this CDR would effectively risk-stratify patients and immediately identify: (A) patients for whom hospitalisation may be safely avoided; and (B) high-risk patients, facilitating judicious use of resources.

Methods

In two sequential prospective observational cohort studies at heterogeneous centres, we included ED patients with suspected cardiac chest pain. We recorded clinical features and drew blood on arrival. The primary outcome was major adverse cardiac events (MACE) (death, prevalent or incident acute myocardial infarction, coronary revascularisation or new coronary stenosis >50%) within 30 days. The CDR was derived by logistic regression, considering reliable (κ>0.6) univariate predictors (p<0.05) for inclusion.

Results

In the derivation study (n=698) we derived a CDR including eight variables (high sensitivity troponin T; heart-type fatty acid binding protein; ECG ischaemia; diaphoresis observed; vomiting; pain radiation to right arm/shoulder; worsening angina; hypotension), which had a C-statistic of 0.95 (95% CI 0.93 to 0.97) implying near perfect diagnostic performance. On external validation (n=463) the CDR identified 27.0% of patients as ‘very low risk’ and potentially suitable for discharge from the ED. 0.0% of these patients had prevalent acute myocardial infarction and 1.6% developed MACE (n=2; both coronary stenoses without revascularisation). 9.9% of patients were classified as ‘high-risk’, 95.7% of whom developed MACE.

Conclusions

The Manchester Acute Coronary Syndromes (MACS) rule has the potential to safely reduce unnecessary hospital admissions and facilitate judicious use of high dependency resources.

Background

Chest pain is the most common reason for emergency hospital admission and the number of admissions is increasing.¹ Only a minority of patients who are admitted to hospital on suspicion that they have an acute coronary syndrome ultimately have that diagnosis.² Improving diagnostic technology to ‘rule out’ ACSs at the time patients first arrive in the emergency department (ED) could (A) reduce unnecessary hospital admissions, with potentially substantial economic benefits for health services and (B) enable patients without serious disease to be given earlier reassurance.

While the imperfect sensitivity and negative predictive value (NPV) of current diagnostic technology at the time of arrival in hospital leads to unnecessary hospital admissions, suboptimal specificity and positive predictive value (PPV) also present challenges. Using a high sensitivity troponin assay, for example, over half of elderly patients who do not have acute myocardial infarction (AMI) will have elevated troponin levels on arrival.³ As further information such as the δ troponin is not available at the time of arrival, there remains a need for novel decision aids that emergency physicians may use to selectively target specialist referral and use of high dependency resources to patients who are at the highest acute risk, enabling judicious use of healthcare resources and avoiding exposure of patients to unnecessary treatment with its consequent risks.

We therefore sought to derive and subsequently externally validate a customised clinical decision rule (CDR) to guide the management of undifferentiated patients presenting to the ED with possible cardiac chest pain. This CDR would combine clinical information and biomarker levels. It would be designed to reduce unnecessary hospital admissions for very low-risk patients and to risk stratify the remaining patients, facilitating judicious resource use.

Methods

This report details the primary analyses from two sequential prospective observational diagnostic cohort studies, first at Manchester Royal Infirmary, an inner city university affiliated teaching hospital (the derivation study) and subsequently at Stepping Hill Hospital, Stockport, a suburban district general hospital (the validation study). A number of secondary analyses have already been published from the derivation study.^4–9

In both studies we included consenting adults aged >25 years who presented to the ED within 24 h of experiencing chest pain suspected to be cardiac in origin by the initial treating physician. We excluded patients with another medical condition requiring hospital admission, renal failure needing dialysis, significant chest trauma with suspicion of myocardial contusion, pregnancy, those who did not speak English, prisoners and those for whom all means of follow-up would be impossible. All participants provided written informed consent and ethical approval for each study was obtained from the Research Ethics Committees (references 05/Q1410/27 and 09/H1014/74).

The treating clinician recorded clinical features and their ECG interpretation using a custom-designed case report form, which included check boxes for all the variables considered for inclusion in the decision rule. Clinical data were recorded at the time of presentation meaning that clinicians were blinded to patient outcome and the results of reference standard investigations. In these observational studies, all patients were managed according to the standard practice at each institution. Interobserver reliability was evaluated in a subgroup of 43 (5.4%) patients and is fully reported elsewhere.⁹ Blood was drawn at the time of presentation for biomarker evaluation. Serum and plasma samples were stored at −70°C pending later analysis. All patients underwent cardiac troponin T testing (cTnT, Roche Diagnostics fourth generation Elecsys, 99th centile 10 ng/L, coefficient of variation <10% at 35 ng/L) on arrival and at least 12 h after symptom onset as part of the reference standard for AMI diagnosis. Patients were followed up after 30 days by telephone, email, letter, home visit or in clinic. If we were unable to complete follow-up by any of these means we contacted the patient's general practitioner.

Laboratory analyses

Aliquots of serum and plasma drawn at the time of arrival in the ED (regardless of the time from symptom onset) were tested for promising cardiac biomarkers that had been identified on literature review. Full details of each assay are described in the online supplementary appendix. In the validation study, admission and 12-h samples were tested for high-sensitivity cTnT (hs-cTnT) although these levels were not available to clinicians or used for clinical decision making during the study period. After the initial analysis, we became aware that the batch of reagents supplied by the manufacturer for measurement of hs-cTnT levels in the validation study had been affected by a calibration shift.¹⁰ Thus, we retested aliquots of serum from all patients using an unaffected batch of reagents and repeated all statistical analyses, which are presented here.

Outcomes

The primary outcome was major adverse cardiac events (MACE) within 30 days. This was a composite of prevalent or incident AMI, death (all cause) or the need for coronary revascularisation (primary percutaneous coronary intervention or coronary artery bypass grafting). In the event that a patient was found to have a significant (>50%) angiographic coronary stenosis (as reported by the treating interventional cardiologist) that was not known to be old, they were also deemed (a priori) to have met the primary outcome. This condition was intended to account for situations where a patient may have significant coronary disease that, because of technical difficulties or excessive risk, was not amenable to revascularisation. The diagnosis of AMI alone (ie, separated from the composite primary outcome) was a secondary outcome.

The diagnosis of AMI was adjudicated by two independent investigators who had all clinical, laboratory and imaging evidence available for review, but who were blinded to investigational biomarker levels. In accordance with the universal definition of AMI,¹¹ patients were required to have a troponin rise and/or fall (≥20 ng/L, based on the analytical characteristics of the assay) with at least one troponin measurement above the 99th centile (10 ng/L) in the appropriate clinical context. Disagreements (n=2) were resolved by discussion and were both explained by human error.

Sensitivity analyses

Since embarking on these studies, high sensitivity troponin assays have been implemented in practice and are now used to make the diagnosis of AMI at many institutions, which may increase the apparent prevalence of AMI.¹² In the validation study, therefore, we accounted for this possibility by analysing admission and 12-h samples for hs-cTnT. Two independent investigators then readjudicated outcomes based on hs-cTnT levels, blinded to the results of the previous adjudication and standard troponin T levels. A rise and/or fall of >9.2 ng/L was considered to be statistically significant.¹³ Disagreements (n=1) were resolved by discussion.

Statistical analysis

Derivation of the CDR

Variables that were predictors of AMI or MACE on univariate analysis by logistic regression (p<0.05) and variables with satisfactory interobserver reliability (κ>0.6) were considered for inclusion in the final model. A summary of the univariate analyses is reported in the online supplementary appendix and additional data are published elsewhere.^{5 9} To limit the potential for overfitting, only biomarkers with an area under the receiver operating characteristic curve of >0.75 were considered for inclusion in the final model.

Potential predictor variables were then entered into a multivariate analysis by forward stepwise logistic regression (p<0.05 for entry, p>0.10 for removal, using SPSS V.20.0; SPSS, Chicago, Illinois, USA). Biomarker levels were entered as continuous variables to avoid the loss of potentially valuable diagnostic information at levels below and above dichotomous cut-offs. Hs-cTnT levels below the limit of blank for the assay (<3 ng/L) were entered as 2 ng/L. We then used the predicted probability of developing the primary outcome (according to the final model) to categorise patients into four risk groups, which were defined a priori according to the disposition decisions available to emergency physicians for this patient group: (1) Very low risk (suitable for discharge); (2) Low risk (suitable for a low dependency environment such as an ED observation unit); (3) Moderate risk (general or acute medical ward); and (4) High-risk (patients who may be considered for immediate referral to a high dependency environment, eg, a coronary care unit). Diagnostic accuracy of the CDR was evaluated in derivation and validation studies by calculation of the areas under the receiver operating characteristic curves (AUCs), sensitivity, specificity, PPV and NPV together with 95% CIs (using MedCalc V.12.4.0.0, Mariakerke, Belgium). Baseline characteristics of each cohort were compared by χ² testing (for independent proportions) or the Mann-Whitney U test (ordinal data) using SPSS V.20.0.

After completion of the validation study, we noted that tobacco smoking had been excluded from the initial multivariate analysis in error. We therefore repeated the multivariate analysis with tobacco smoking included. The resultant alternative decision rule and its performance are presented in the online supplementary appendix.

Sample size

Derivation study

Methodological guidance for the derivation of CDRs states that approximately 10 subjects with the primary outcome are required for every predictor included in the decision rule.¹⁴ Based on the numbers of predictive variables identified by previous studies of patients with chest pain^{15 16} and practical considerations, we felt it was unlikely that the final decision rule would incorporate more than 15 predictors, thus requiring 150 patients with the primary outcome to provide appropriate statistical power. We estimated a 20% incidence of the primary outcome^{17 18} and predicted 5% loss to follow-up, which would require a total of 790 participants.

Validation study

To determine the sample size for the validation study, we determined the number of patients that would be required to ensure that, if the CDR was shown to have a sensitivity of 100%, the 95% CI would extend no lower than 95%. Assuming a 20% incidence of the primary outcome, a sample size of 450 participants would ensure this degree of precision.

Results

In total 698 patients were included in the derivation study between January 2006 and February 2007, all of whom completed follow-up (figure 1). Of these patients 130 (18.6%) had AMI and 157 (22.5%) developed the primary outcome (MACE within 30 days). The validation study included a total of 463 patients between April and July 2010, all of whom completed follow-up (figure 1). Of those patients, 79 (17.1%) had AMI and 98 (21.2%) developed MACE within 30 days. Baseline characteristics for each study are shown in table 1.

Figure 1.

Patient flow diagram.

Table 1.

Baseline characteristics of included patients

Variable	Derivation study (N=698)	Validation study (N=463)	p Value
Age in years, mean (SD)	58.6 (14.3)	64 (15)
Men (%)	429 (61.5)	270 (58.3)	0.303
Previous angina (%)	220 (31.5)	186 (40.2)	0.002
Previous myocardial infarction (%)	166 (23.8)	139 (30.0)	0.023
Hypertension (%)	341 (48.9)	197 (42.5)	0.037
Hyperlipidaemia (%)	337 (48.3)	186 (40.2)	0.008
Diabetes mellitus (%)	125 (17.9)	80 (17.3)	0.854
Smoking (%)	216 (30.9)	96 (20.7)	<0.001
Family history of coronary heart disease (%)	337 (48.3)	171 (36.9)	<0.001
Previous coronary intervention (%)	140 (20.1)	102 (22.0)	0.480
Peripheral vascular disease (%)	13 (1.9)	15 (3.2)	0.224
Cerebrovascular disease (%)	71 (10.2)	30 (6.5)	0.037
Time from symptom onset
0–3 h	324 (46.4)	212 (45.8)	0.079
3–6 h	166 (23.8)	94 (20.3)
6–12 h	148 (21.2)	64 (13.8)
>12 h	60 (8.6)	93 (20.1)

Derivation of the Manchester Acute Coronary Syndromes rule

A summary of the univariate analyses is shown in the online supplementary appendix. The final decision rule derived by multivariate analysis included eight variables (table 2). In the derivation study the rule had an AUC of 0.95 (95% CI 0.93 to 0.97) for predicting MACE. Based on the estimated probability assigned by the model, The MACS rule stratified patients into four risk groups (figure 2). The MACS rule successfully stratified patients according to their risk of MACE and AMI. Its use could have enabled the 35.5% of patients in the MACS rule ‘very low risk’ group to be immediately discharged with zero (0.0%) missed AMIs and one (0.4%) missed MACE (a patient with normal ECG and normal troponin who underwent inpatient coronary angiography and percutaneous coronary intervention to lesions in the left anterior descending and circumflex arteries). Thus, as a dichotomous tool for guiding the decision about whether to admit or discharge patients, the MACS rule had an overall sensitivity of 99.4% (95% CI 96.5% to 100.0%) and NPV 99.6% (97.8% to 100.0%) for MACE. For AMI, sensitivity was 100.0% (97.2% to 100.0%) and NPV 100.0% (98.5% to 100.0%). Of the patients in the ‘high-risk’ group (10.6% of the entire cohort) 100.0% developed MACE and 98.6% had AMI.

Table 2.

Multivariate model detailing the components of the MACS rule (rounded values are presented)

Variable	Constant	OR (95% CI)	p Value
a. High sensitivity troponin T*	0.068	1.1 (1.0 to 1.1)	<0.0001
b. Heart-type fatty acid binding protein*	0.17	1.2 (1.0 to 1.4)	0.018
c. ECG ischaemia	1.75	5.8 (3.1 to 10.8)	<0.0001
d. Sweating observed	1.85	6.3 (3.0 to 13.3)	<0.0001
e. Vomiting	1.72	5.6 (1.9 to 16.6)	0.002
f. Systolic blood pressure <100 mm Hg	1.46	4.3 (1.2 to 15.1)	0.022
g. Worsening angina	0.92	2.5 (1.2 to 5.2)	0.014
h. Pain radiating to right arm or shoulder	0.87	2.4 (1.0 to 5.6)	0.044
Constant	−4.83	–	–

The model estimates the probability (p) of MACE as follows (rounded values are presented): p=1/(1+e^{−(0.068a + 0.17b + 1.75c + 1.85d + 1.72e + 1.46f + 0.92g + 0.87h − 4.83)}). For each categorical variable, a value of 1 is assigned if the characteristic is present and 0 if absent.

*ORs are presented for a 1 unit increase (ng/L for hs-cTnT; ng/mL for H-FABP).

MACE, major adverse cardiac events; MACS, Manchester Acute Coronary Syndromes.

Figure 2.

Performance of the Manchester Acute Coronary Syndromes decision rule in the derivation study. The suggested disposition for each group is as follows: ‘Very low risk’—discharge immediately (estimated probability, p, of MACE <0.02); ‘low risk’—admit to low dependency environment, for example, emergency department observation unit, 0.02≤p<0.05); ‘moderate risk’—admit to acute ward, for example, medical admissions unit (0.05≤p<0.95); ‘high risk’—admit to coronary care unit or high dependency environment (p≥0.95). AMI, acute myocardial infarction; MACE, major adverse cardiac events.

External validation

The performance of the MACS rule on external validation is shown in figure 3. Again, the MACS rule effectively stratified patients according to their risk of MACE and AMI. The rule had an AUC of 0.92 (95% CI 0.89 to 0.95) for MACE. The 125 (27.0%) patients in the ‘very low risk’ group could have been immediately discharged with zero (0.0%) missed AMIs and two (1.6%) MACEs within 30 days. Both of these events were coronary stenoses detected at angiography but that were not deemed to warrant coronary intervention. Thus the MACS rule would have enabled decisions to admit or discharge patients to have been made with a sensitivity of 98.0% (95% CI 93.0% to 99.8%) and NPV 98.7% (95.3% to 99.8%) for MACE. For AMI, the MACS rule had a sensitivity of 100.0% (95% CI 95.4% to 100.0%) and NPV 100.0% (97.1% to 100.0%). Of the ‘high-risk’ patients (n=46, 9.9% of the entire cohort) 95.7% developed MACE and 91.3% had AMI.

Figure 3.

Performance of the Manchester Acute Coronary Syndromes decision rule in the validation study. Both MACEs in the very low risk group were coronary stenoses identified on an outpatient basis that did not require intervention. AMI, acute myocardial infarction; MACE, major adverse cardiac events.

Sensitivity analyses

With a readjudicated diagnosis of AMI using hs-cTnT as the reference standard, the rates of MACE and AMI stratified by MACS rule risk group are shown in figure 4. Again, two (1.6%) patients in the very low risk group developed MACE, which were both coronary stenoses not deemed to warrant revascularisation. The AUC was 0.92 (95% CI 0.88 to 0.95). As a dichotomous tool for facilitating immediate hospital discharge, the MACS rule had an overall sensitivity of 98.4% (94.3% to 99.8%) and NPV 97.9% (92.8% to 99.8%) for MACE. For AMI, the sensitivity was 100.0% (95.3% to 100.0%) and NPV 100.0% (97.0% to 100.0%).

Figure 4.

Performance of the Manchester Acute Coronary Syndromes decision rule with high-sensitivity cardiac troponin T (hs-cTnT) as the reference standard for the adjudicated diagnosis of AMI (validation study). A total of 77 patients (16.6%) were given a readjudicated diagnosis of AMI and 97 (21.0%) developed MACE. AMI, acute myocardial infarction; MACE, major adverse cardiac events.

Discussion

We have presented the derivation and subsequent external validation of the MACS CDR for use in patients presenting to hospital with suspected cardiac chest pain, which could potentially obviate the need for emergency hospital admission for over a quarter of patients following a single blood test. This has great potential benefit for patients, who could be immediately reassured that they do not have AMI and that any appropriate follow-up can safely be arranged on an outpatient basis. Given the large number of patients affected (over a quarter of acute medical admissions), use of the MACS rule may help to relieve the increasing pressure on EDs, where overcrowding is associated with more patient safety incidents and rising mortality.¹⁹ The potential cost saving for health services is also substantial.

In addition to identifying very low risk patients for whom unnecessary hospital admission may be avoided, the MACS rule risk stratifies the remaining patients. While the low and moderate risk patients still require further inpatient investigation before an acute coronary syndrome can be confirmed or excluded, patients in the high-risk group had a very high prevalence of AMI and incidence of MACE, potentially facilitating selective specialist referral and targeting of high dependency resources with high specificity and PPV.

A number of alternative ‘early rule out’ strategies have been reported. Several CDRs have been derived for use in the ED population, although their usefulness is limited by low specificity and none were derived in the era of high sensitivity troponins.^{20 21} Serial biomarker testing over 60–120 min could facilitate safe early discharge of low-risk patients with chest pain,^22–25 although such strategies may not always be cost-effective.²⁶ Using high sensitivity troponin alone it may be possible to exclude AMI within 60–180 min of presentation.^{27 28} A secondary analysis from our cohort suggested that a single hs-cTnT level below the limit of blank may be sufficient to exclude AMI, although this would not facilitate the immediate discharge of as many patients as the MACS rule and does not make use of clinical, ECG and biochemical data to optimise risk stratification.⁴ Finally, the HEART score is an emerging alternative that may enable the safe early discharge of a proportion of patients presenting to the ED.²⁹ Prospective comparison of the MACS rule with the most promising of these emerging technologies is an important area for future research.

Strengths and limitations

This MACS rule was derived in accordance with established methodological guidance¹⁴ and provides a bespoke tool to aid clinical decision making specifically in the undifferentiated ED population. It has the potential to reduce unnecessary hospital admissions by facilitating the immediate discharge of over a quarter of patients using a single blood test at the time of arrival. The MACS rule also effectively risk stratifies the remaining patients, identifying high-risk patients with high PPV and suggesting courses of action to the clinician. We hope that this will prevent a rebound overuse of resources, as has happened with previous rapid rule-out strategies that have high sensitivity but relatively low specificity.²⁶

The MACS rule has several novel features. It combines clinical and biochemical data and incorporates contemporary biomarker levels, which are considered as continuous variables. This maximises the richness of diagnostic information that may be lost by dichotomisation and enables levels even below the 99th centile to influence the initial risk stratification, building and improving upon a previous analysis from our derivation cohort.⁴

We also consider that inclusion of a second prospective external validation study is a strength of this report. This work enabled the performance of the MACS rule to be verified in a heterogeneous (as demonstrated in table 1) and independent population, reducing the potential for statistical overfitting and overestimation of diagnostic performance.

There are some limitations to our work. We aimed to derive and validate a CDR that could be used to guide decisions made in the ED at the time of initial presentation or following a single blood test. However, with the addition of serial sampling at 60–90 min it may have been possible to safely identify an even greater proportion of patients as eligible for safe early discharge. This strategy has shown promise when used with biomarker panels and serial troponin testing, particularly in low-risk patients.^{22 23 27} Future work should explore whether refinements to include early serial sampling will enhance diagnostic performance of the MACS rule. Comparison with emerging alternatives such as the HEART score will also be important.

It is important to acknowledge that the MACS rule applies to the specific biomarkers and clinical observations used. Future assessment with even larger cohorts and other combinations of newer biomarkers or observations might, in due course, enable further refinement and stratification of the clinical decision-making process in this group of patients.

Finally, the MACS rule requires a complex calculation necessitating the use of a computer. Rather than categorising or dichotomising variables, we have opted to retain the richness of our data to optimise diagnostic performance. Other computerised clinical prediction models are in widespread use in current clinical practice. The Global Registry of Acute Coronary Events score is a good example.³⁰ As it is almost certain that clinicians practising in a contemporary ED with access to contemporary biomarker assays will have access to computers in order to undertake the required calculation, we do not consider that this will limit the usefulness of the MACS decision rule in a clinical setting.

Future directions

Having derived and successfully externally validated the MACS rule in two sequential observational studies, it is now imperative to further evaluate its impact when used in practice. Use of the MACS rule may lead to changes in the subsequent investigations and treatment that patients receive, which may affect outcome rates. Compliance with the decision rule by physicians and patients is also unlikely to be perfect. Furthermore, use of the rule may still have unanticipated effects such as increasing actual resource utilisation.²⁶ A randomised controlled trial is therefore warranted prior to widespread implementation, with analyses focusing on clinical, economic and process outcomes.

Conclusions

We have derived and externally validated the MACS CDR for use in patients presenting to the ED with suspected cardiac chest pain. Over 25% of patients could be immediately discharged, reducing unnecessary admissions, while selectively identifying high-risk patients to make judicious use of specialist resources. It is now important to evaluate the clinical and cost-effectiveness of the MACS rule when it is used in clinical practice.

Key messages.

What is already known on this subject?

• Current diagnostic technology, including high sensitivity troponin assays, has imperfect sensitivity and specificity for acute coronary syndromes at the time of initial presentation to the emergency department. There is a need for diagnostic tools that may help to reduce unnecessary hospital admissions while making judicious use of specialist resources.

What this study adds?

• Using information available at the time of first presentation and results from a single blood test, the Manchester Acute Coronary Syndromes rule could enable more than a quarter of patients to be immediately discharged. The rule also selectively identifies high-risk patients, which may help emergency physicians to target specialist referral and use of high dependency resources to those patients at highest risk.

How might this impact on clinical practice?

• Chest pain is the most common reason for emergency hospital admission in England and Wales. A decision rule that reduces the need for unnecessary hospital admission would have tremendous economic benefits for the Health Service while reducing inconvenience and exposure to the risks of unnecessary treatment for patients.