Abstract
Background:
Syncope can be caused by serious conditions not evident during initial evaluation, which can lead to serious adverse events, including death, after disposition from the emergency department. We sought to develop a clinical decision tool to identify adult patients with syncope who are at risk of a serious adverse event within 30 days after disposition from the emergency department.
Methods:
We prospectively enrolled adults (age ≥ 16 yr) with syncope who presented within 24 hours after the event to 1 of 6 large emergency departments from Sept. 29, 2010, to Feb. 27, 2014. We collected standardized variables at index presentation from clinical evaluation and investigations. Adjudicated serious adverse events included death, myocardial infarction, arrhythmia, structural heart disease, pulmonary embolism, serious hemorrhage and procedural interventions within 30 days.
Results:
We enrolled 4030 patients with syncope; the mean age was 53.6 years, 55.5% were women, and 9.5% were admitted to hospital. Serious adverse events occurred in 147 (3.6%) of the patients within 30 days after disposition from the emergency department. Of 43 candidate predictors examined, we included 9 in the final model: predisposition to vasovagal syncope, heart disease, any systolic pressure reading in the emergency department < 90 or > 180 mm Hg, troponin level above 99th percentile for the normal population, abnormal QRS axis (< −30° or > 100°), QRS duration longer than 130 ms, QTc interval longer than 480 ms, emergency department diagnosis of cardiac syncope and emergency department diagnosis of vasovagal syncope (C statistic 0.88, 95% confidence interval [CI] 0.85–0.90; optimism 0.015; goodness-of-fit p = 0.11). The risk of a serious adverse event within 30 days ranged from 0.4% for a score of −3 to 83.6% for a score of 11. The sensitivity was 99.2% (95% CI 95.9%–100%) for a threshold score of −2 or higher and 97.7% (95% CI 93.5%–99.5%) for a threshold score of −1 or higher.
Interpretation:
The Canadian Syncope Risk Score showed good discrimination and calibration for 30-day risk of serious adverse events after disposition from the emergency department. Once validated, the tool will be able to accurately stratify the risk of serious adverse events among patients presenting with syncope, including those at low risk who can be discharged home quickly.
Syncope is common both in the general population and in acute care settings, accounting for 1%–3% of all emergency department visits.1–3 It is a transient loss of consciousness due to transient global cerebral hypoperfusion followed by spontaneous complete recovery.4 Although often benign, syncope can be caused by potentially life-threatening conditions such as ventricular arrhythmias.5 These serious conditions and the associated morbidity and mortality constitute serious adverse events among patients with syncope. A substantial proportion of serious adverse events are not apparent during initial evaluation in the emergency department.6 From 7% to 23% of patients with syncope will experience a serious adverse event within 7–30 days, with about half being evident only after disposition from the emergency department, either while admitted in hospital or after discharge home.7–9
The proportion of patients with syncope who are admitted to hospital, and the evaluation and investigations ordered both in the emergency department and in hospital vary greatly among physicians, institutions and countries.1,10,11 A recent study showed wide variations in the management of syncope in emergency departments, and poor agreement between disposition decisions by physicians and significant differences in the 30-day rate of serious outcomes across the study centres.11 Risk stratification and disposition of patients with syncope is challenging because valid and reliable evidence guiding disposition decisions is lacking.12 A few prospective studies developed clinical decision tools to identify patients at risk of serious adverse events within 30 days.8,9,13,14 However, they have poor diagnostic test characteristics and several methodological flaws that preclude their widespread adoption by clinicians.
We conducted a prospective cohort study to develop a clinical decision tool to identify adult patients with syncope who are at risk of a serious adverse event within 30 days after disposition from the emergency department.
Methods
Study setting and population
We conducted a prospective cohort study at 6 large emergency departments in teaching hospitals in 4 Canadian cities (The Ottawa Hospital–Civic and General Campuses, Ottawa; the Kingston General Hospital and the Hotel Dieu Hospital, Kingston, Ontario; the Foothills Medical Centre, Calgary; and the University of Alberta Hospital, Edmonton).
We included adults (age ≥ 16 yr) with syncope who presented within 24 hours after the event from Sept. 29, 2010, to Feb. 27, 2014. We excluded patients who had prolonged loss of consciousness (> 5 min), a change in their mental status from baseline after the syncope, an obvious witnessed seizure, major trauma requiring hospital admission, intoxication with alcohol or illicit drugs, a language barrier or head trauma causing loss of consciousness. We chose these exclusion criteria because they are routinely used in syncope studies to exclude patients without true syncope.4,15 We also excluded patients who had a serious adverse event identified during the index emergency department visit. Because of wide variation in the disposition of patients with syncope at the study sites,11 we included both admitted and discharged patients in our analysis to avoid bias.
Because our study was observational, the ethics committees at all of the sites approved the protocol with the requirement of only verbal consent.
Data collection
Emergency physicians and emergency medicine residents under staff physicians’ supervision screened consecutive patients, confirmed eligibility and obtained consent before inclusion in the study. The physicians and residents attended a 1-hour training session on how to assess for standardized variables from the history and physical examination, and how to arrive at the final diagnosis based on the European Society of Cardiology guidelines.4 All electrocardiograms (ECGs) performed during the index visit to the emergency department were reviewed by at least 1 cardiologist; ECGs with abnormalities were reviewed by a study cardiologist for extraction of ECG variables.16
We chose the list of variables for data collection on the basis of a comprehensive literature review, the results of previously completed studies and expert consensus.15,17–19 We identified a subset of these variables as candidate predictors for the study. When feasible, a second emergency physician assessed a subset of study patients to estimate interobserver agreement of the eligibility criteria and predictor variables. Trained research assistants reviewed the presenting problems and discharge diagnoses for all visits to the participating emergency departments during the study period to identify additional patients who would have been eligible for the study but were not enrolled.
Serious adverse events
We defined a serious adverse event as the detection or occurrence of any serious condition related to syncope within 30 days after disposition from the emergency department. The composite outcome included any of the following serious adverse events: death, arrhythmia, myocardial infarction, serious structural heart disease, aortic dissection, pulmonary embolism, severe pulmonary hypertension, severe hemorrhage, subarachnoid hemorrhage, any other serious condition causing syncope and procedural interventions for the treatment of syncope (details are provided in Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151469/-/DC1). These adverse events have been identified by 2 international panels of syncope researchers as pertinent ones that should be reported in risk-stratification studies of syncope management in emergency departments in an effort to standardize the reporting of outcomes.15,20 For patients who experienced a serious adverse event, we collected information on the place of the occurrence (inside or outside the hospital).
We assessed for the composite outcome using a multistep approach: (a) a structured review of patients’ health records for documents related to the index and subsequent visits to the emergency department; inpatient, outpatient and operative notes; results of all investigations; and hospital death records; (b) scripted telephone follow-up immediately after 30 days; (c) at the Ontario study sites, a review of patients’ health records at all local adult hospitals and the provincial coroner’s office for death records; and (d) at the Alberta study sites, a review of the Alberta health systems database (Netcare) to ascertain return visits to the emergency department. All serious adverse events were confirmed by an adjudication committee composed of 2 physicians blinded to the predictors and a third physician who adjudicated in cases of disagreement.
Statistical analysis
We describe study participants using means, ranges and standard deviations for continuous variables, and frequencies with proportions for categorical variables. The interobserver agreement was calculated with the use of the κ coefficient, the proportion of agreement beyond chance.
We screened the initial list of variables of interest to identify a subset of candidate predictors for bivariable and multivariable analysis. We excluded candidate predictors with fewer than 5 expected events because they were likely to cause model instability; those with variance inflation factors for multicollinearity exceeding a threshold of 5; those with missing values for more than 25% of the patients; and those with unacceptable interobserver agreement (κ value < 0.4).21 Because there were more candidate predictors than available degrees of freedom, we selected candidate predictors for multivariable logistic regression by testing the bivariable association of each predictor with the outcome at the 5% significance level, using the χ2 or Fisher exact test for categorical predictors and the 2-sample t test for continuous predictors. Before the regression analysis, we performed multiple imputation for missing predictors to create a complete data set for analysis. Ten multiple imputation data sets were generated with the use of the Markov chain Monte Carlo method with inclusion of the outcome, all candidate predictors and additional variables expected to be correlated with the missing predictors.22
After multiple imputation, we dichotomized continuous predictors using a combination of clinical rationale and analysis of receiver operating characteristic curves, which identified the optimal cut-off point based on measures of sensitivity, specificity and the Youden Index.12,17,18 After categorization, predictors selected by bivariable analysis were included in a multivariable logistic regression model, which was reduced by stepwise backward elimination with a 5% significance level to stay in the model. The regression estimates for the reduced model were combined across the 10 multiple imputation data sets.22
Internal validation was carried out with the use of 500 bootstrap samples. The variable selection procedure was repeated in each bootstrap sample, and the percentage of times each variable was selected was determined to examine the stability of the stepwise variable selection procedure. A well-known problem in multivariable regression modelling is overfitting — the model performs well for predicting outcomes among the patients used to derive the model but poorly among new patients.21 This leads to an overly optimistic impression of the model’s performance. The optimism (overfitting) in our model performance measures was estimated, and optimism-corrected performance indicators with 95% bootstrap confidence intervals (CIs) were obtained. As a result of overfitting, predictions tend to be too low for patients with low risk scores and too high for those with high risk scores. A solution is to apply shrinkage, which adjusts estimated regression coefficients to be closer to 0. To correct for overfitting, we calculated the model shrinkage using the calibration slopes across the bootstrap samples and applied it to the regression coefficients.23
We translated the shrinkage-corrected model into a point-scoring system by dividing all of the regression coefficients by the smallest coefficient and rounding to the nearest integer.24 We assessed the calibration of the model using a comparison of observed versus expected risk at each level of the score, as well as the Hosmer–Lemeshow goodness-of-fit statistic by risk percentiles. We also tested the ability of the model to discriminate the 4 subtypes of serious adverse event (death, arrhythmia, nonarrhythmia cardiac, and noncardiac) that form the composite outcome.
We used SAS versions 9.3 and 9.4 for data analysis.
Sample size
We determined the sample size based on the estimation of precision of the sensitivity of the prediction tool to be developed in the study population.25 We determined that 125 patients with syncope who had a serious adverse event within 30 days after disposition from the emergency department would be required to achieve a target of 99% sensitivity with a 95% exact binomial CI of 95.6%–100%. Conservatively assuming a prevalence of 3.5% for serious adverse events after disposition, we calculated a total required sample size of 3571 patients.
Results
We enrolled 4322 patients with syncope at the study hospitals from Sept. 29, 2010, to Feb. 27, 2014 (Figure 1). We excluded 292 (6.8%) because they had incomplete outcome assessments (they could not be contacted for follow-up by telephone or had no matching records in the databases searched), which left 4030 for analysis. Of these patients, we were able to achieve follow-up by direct telephone contact with 3855 (95.7%) or their family; for the remaining 175 patients (4.3%), we were able to ascertain positive or negative outcome status by reviewing hospital health records, coroner’s records or the health systems database. The demographic characteristics of the 1263 (20.5%) potentially eligible patients who were not enrolled were similar to those of the enrolled patients (mean age 56.0 v. 53.6 yr; 54.1% v. 55.5% females). Interrater reliability assessments were performed by a second physician for 190 (4.7%) of the 4030 patients, and the κ value for syncope confirmation and inclusion in the study was 0.76 (95% CI 0.64–0.87).
Figure 1:
Selection of patients for the study.
Table 1 shows the characteristics of the 4030 patients, their management in the emergency department and their outcomes. Within the 30-day follow-up period, 147 patients (3.6%, 95% CI 3.1%–4.3%) experienced a serious adverse event after disposition from the emergency department, of whom 61 (1.5% [41.5% of those with serious adverse events]) had it outside the hospital (Table 2).
Table 1:
Characteristics, emergency department management and outcomes of patients presenting with syncope
| Variable | No. (%) of patients n = 4030 |
|---|---|
| Age, yr | |
| Mean ± SD | 53.6 ± 23.0 |
| Range | 16–102 |
| Sex | |
| Female | 2238 (55.5) |
| Male | 1792 (44.5) |
| Arrival by ambulance | |
| Yes | 2590 (64.3) |
| No | 1440 (35.7) |
| Medical history | |
| Hypertension | 1292 (32.1) |
| Diabetes mellitus | 402 (10.0) |
| Coronary artery disease | 476 (11.8) |
| Atrial fibrillation or flutter | 291 (7.2) |
| Valvular heart disease | 137 (3.4) |
| Congestive heart failure | 152 (3.8) |
| Management in emergency department | |
| Electrocardiography | 3834 (95.1) |
| Blood tests | 3446 (85.5) |
| Admission to hospital | 381 (9.5) |
| Outcome | |
| Serious adverse event in hospital | 86 (2.1) |
| Serious adverse event outside hospital | 61 (1.5) |
Table 2:
Serious adverse events among patients with syncope after disposition from emergency department
| Serious adverse event | Location of adverse event; no. of patients | ||
|---|---|---|---|
| Total n = 147 |
In hospital n = 86 |
Outside hospital n = 61 |
|
| Death | 21 | 8 | 13 |
| Known cause* | 7 | 5 | 2 |
| Unknown cause | 14 | 3† | 11 |
| Cardiac | 98 | 65 | 33 |
| Arrhythmia | |||
| Sinus node dysfunction | 18 | 10 | 8 |
| New or uncontrolled atrial fibrillation | 7 | 0 | 7 |
| High-grade atrioventricular block | 9 | 6 | 3 |
| Ventricular arrhythmia | 15 | 11 | 4 |
| Supraventricular tachycardia | 2 | 1 | 1 |
| Pacemaker insertion | 24 | 21 | 3 |
| Other cardiac | |||
| Structural heart disease | 11 | 9 | 2 |
| Myocardial infarction | 11 | 6 | 5 |
| Aortic dissection | 1 | 1 | 0 |
| Noncardiac | 35 | 18 | 17 |
| Pulmonary embolism | 8 | 6 | 2 |
| Gastrointestinal bleeding | 10 | 6 | 4 |
| Other‡ | 17 | 6 | 11 |
Patients with a known cause of death had one of the cardiac or noncardiac serious adverse events and hence are not counted toward the total number of patients with serious outcomes.
Multiple causes of death were reported in the medical records, with no clear indication of cause of death.
Includes conditions that require acute treatment or that cause a patient to return to hospital if discharged home, such as renal failure requiring dialysis, sepsis, anemia requiring transfusion, acute abdominal conditions (i.e., bowel obstruction, acute appendicitis, ectopic pregnancy, acute presentation of intra-abdominal or pelvic cancer, and cholangitis), lung pathology (i.e., severe pulmonary hypertension, pulmonary fibrosis, and large pleural effusion), acute intracranial pathology (i.e., subarachnoid hemorrhage, posterior circulation stroke, brain tumour, brain metastasis, intracranial hemorrhage, subdural hematoma) and subclavian steal syndrome.
We selected an initial list of 43 candidate predictors for analysis (Appendix 2, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151469/-/DC1). Of these, 10 were excluded because of sparse distributions; none was excluded because of a low κ value or large proportion of missing values. For troponin levels, assays performed at the study sites differed, and the values were not comparable. In addition, troponin levels were not measured in 52.1% of patients. We investigated the plausibility of the assumption that patients with missing values would be within the normal range by comparing the characteristics of patients with and without troponin measurements; we found that those whose levels were not measured were younger, had a lower prevalence of comorbidities and had a lower proportion who experienced a serious adverse event (Appendix 3, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151469/-/DC1). We therefore dichotomized troponin at the 99th percentile cut-off value for the normal population and assumed that all missing values were within the normal range. Bivariable tests of association were performed for the remaining 33 candidate predictors. A total of 8 predictors were not significant, and after categorization, 2 additional predictors (triage oxygen saturation and hemoglobin) had very low prevalence at abnormal cut-off values. We excluded these 10 predictors from further consideration.
A final list of 23 candidate predictors was selected for multivariable modelling, and none was involved in near linear dependencies (Table 3). After exclusion of the predictor troponin, the percentage of missing values for the remaining candidate predictors ranged from 0% to 23.8%. Data for 4 predictors were missing for more than 10% of patients: urea (23.8%), triage respiratory rate (20.2%), creatinine (18.1%) and hematocrit (16.4%). The predictor history of heart disease was missing for 6.3% of patients, and the ECG predictors were missing for 4.9% of patients. The remaining predictors were missing for less than 2% of patients. A total of 1887 participants (46.8%) had missing values for at least 1 of the 22 predictors, most of whom were in the group with no serious adverse event (Table 3). After multiple imputation, all 4030 participants were included in the analysis. The 12 continuous predictors were dichotomized; after stepwise backward elimination, the results were combined across the multiple imputation data sets to account for imputation uncertainty.
Table 3:
Association between predictors and occurrence of serious adverse events
| Predictor* | Outcome; no. (%) or mean ± SD | p value | κ value† | |
|---|---|---|---|---|
| Serious adverse event n = 147 |
No serious adverse event n = 3883 |
|||
| Demographic | ||||
| Age, yr | 71.6 ± 16.7 | 52.9 ± 23.0 | < 0.001 | – |
| Female sex | 63 (42.9) | 2175 (56.0) | 0.002 | – |
| Medical history | ||||
| Vascular disease‡ (n = 147; 3876) | 16 (10.9) | 238 (6.1) | 0.02 | 0.85 |
| Heart disease§ (n = 138; 3639) | 83 (60.1) | 748 (20.6) | < 0.001 | 0.95 |
| Event details | ||||
| Predisposition to vasovagal syncope¶ (n = 147; 3820) | 19 (12.9) | 1642 (43.0) | < 0.001 | 0.48 |
| Presence of prodrome** (n = 144; 3817) | 82 (56.9) | 2903 (76.1) | < 0.001 | 0.57 |
| Vital sign in emergency department | ||||
| Triage systolic BP (n = 142; 3769) | 133.3 ± 33.0 | 125.3 ± 21.4 | 0.005 | – |
| Highest systolic BP (n = 147; 3853) | 149.1 ± 29.5 | 137.1 ± 22.1 | < 0.001 | – |
| Lowest diastolic BP (n = 146; 3851) | 61.6 ± 17.3 | 65.3 ± 12.8 | 0.01 | – |
| Highest diastolic BP (n = 147; 3850) | 83.6 ± 17.6 | 80.4 ± 13.6 | 0.03 | – |
| Highest heart rate (n = 147; 3851) | 87.2 ± 19.7 | 84 ± 16.6 | 0.05 | – |
| Triage respiratory rate (n = 119; 3099) | 18.0 ± 3.8 | 17.2 ± 2.6 | 0.04 | – |
| Laboratory value | ||||
| Elevated troponin (> 99th percentile of normal population) | 37 (25.2) | 132 (3.4) | < 0.001 | – |
| Hematocrit (n = 145; 3225) | 0.38 ± 0.06 | 0.40 ± 0.05 | < 0.001 | – |
| Serum urea, mmol/L (n = 134; 2939) | 8.8 ± 6.8 | 6.4 ± 4.4 | < 0.001 | – |
| Serum creatinine, μmol/L (n = 145; 3157) | 109.5 ± 68.2 | 88.4 ± 66.0 | < 0.001 | – |
| ECG evidence (n = 143; 3691) | ||||
| Left bundle branch block | 15 (11.4) | 85 (2.3) | < 0.001 | 0.88 |
| Left ventricular hypertrophy | 13 (9.9) | 199 (5.5) | 0.03 | 0.83 |
| QRS duration, ms | 111.1 ± 31.7 | 93.0 ± 17.6 | < 0.001 | – |
| QRS axis, degrees | 16.0 ± 60.4 | 35.6 ± 42.3 | < 0.001 | – |
| Corrected QT interval, ms | 463.6 ± 43.3 | 431.7 ± 30.7 | < 0.001 | – |
| Diagnosis in emergency department | ||||
| Vasovagal syncope | 19 (13.0) | 2165 (55.8) | < 0.001 | 0.60 |
| Cardiac syncope | 51 (34.9) | 194 (5.0) | < 0.001 | 0.60 |
| Categorization of continuous predictors | ||||
| Age > 75 yr | 72 (49.0) | 871 (22.4) | < 0.001 | – |
| Any systolic BP < 90 or > 180 mm Hg†† (n = 147; 3871) | 46 (31.3) | 446 (11.5) | < 0.001 | – |
| Lowest diastolic BP < 50 mm Hg | 30 (20.6) | 348 (9.0) | < 0.001 | – |
| Highest diastolic BP > 110 mm Hg | 12 (8.2) | 90 (2.3) | < 0.001 | – |
| Highest heart rate > 110 beats/min | 18 (12.2) | 244 (6.3) | 0.004 | – |
| Triage respiratory rate > 20 breaths/min | 12 (10.1) | 109 (3.5) | < 0.001 | – |
| Hematocrit < 0.3 | 13 (9.0) | 113 (3.5) | < 0.001 | – |
| Urea > 12 mmol/L | 21 (15.7) | 164 (5.6) | < 0.001 | – |
| Creatinine > 150 μmol/L | 17 (11.7) | 170 (5.4) | 0.001 | – |
| QRS duration > 130 ms | 42 (31.6) | 186 (5.1) | < 0.001 | – |
| Abnormal QRS axis (< −30° or > 110°) | 43 (32.3) | 307 (8.3) | < 0.001 | – |
| Corrected QT interval > 480 ms | 51 (38.4) | 236 (6.4) | < 0.001 | – |
Note: BP = blood pressure, ECG = electrocardiogram.
Numbers within brackets indicate the number of patients in each group for whom data were available for the variable. Where numbers are not reported, the data were available for all of the patients.
Interobserver assessments were conducted for data from 190 (4.7%) of the 4030 patients.
Includes transient ischemic attack, cerebrovascular accident and peripheral vascular disease.
Includes coronary or valvular heart disease, cardiomyopathy, congestive heart failure and non-sinus rhythm (ECG evidence during index visit or documented history of ventricular or atrial arrhythmias, or device implantation).
Triggered by being in a warm crowded place, prolonged standing, fear, emotion or pain.
Dizziness, light-headedness, vision changes, nausea or vomiting.
Includes BP values from triage until disposition from emergency department.
The final model included 9 predictors (Table 4). Three predictors were from clinical evaluation (predisposition to vasovagal syncope; history of heart disease; and any systolic pressure reading in the emergency department < 90 or > 180 mm Hg). Four predictors were from investigations (troponin level > 99th percentile for normal population; and 3 ECG predictors [abnormal QRS axis, prolonged QRS interval and prolonged corrected QT interval]). The final 2 predictors were related to emergency department diagnosis (vasovagal or cardiac syncope). The apparent C statistic for the model was 0.88 (95% CI 0.85–0.90); after accounting for optimism of 0.015, the optimism-corrected C statistic was 0.87 (95% CI 0.84–0.89). The model shrinkage factor was calculated as 0.91 using the bootstrap internal validation, which indicated that about 9% of the apparent model performance can be attributed to statistical overfitting.
Table 4:
Multivariable logistic regression model used to derive the Canadian Syncope Risk Score
| Variable | β coefficient* | p value | Odds ratio* (95% CI) |
|---|---|---|---|
| Predisposition to vasovagal syncope† | −0.56 | 0.03 | 0.57 (0.33–0.98) |
| History of heart disease‡ | 0.52 | 0.007 | 1.69 (1.13–2.53) |
| Any systolic BP reading < 90 or > 180 mm Hg in emergency department§ | 0.76 | < 0.001 | 2.14 (1.43–3.21) |
| Elevated troponin (> 99th percentile of normal population) | 1.15 | < 0.001 | 3.19 (1.98–5.12) |
| Abnormal QRS axis (< −30° or > 110°) | 0.48 | 0.03 | 1.62 (1.01–2.62) |
| QRS duration > 130 ms | 0.64 | 0.01 | 1.89 (1.11–3.24) |
| Corrected QT interval > 480 ms | 0.90 | < 0.001 | 2.45 (1.54–3.92) |
| Emergency department diagnosis of vasovagal syncope | −0.98 | < 0.001 | 0.38 (0.22–0.66) |
| Emergency department diagnosis of cardiac syncope | 1.09 | < 0.001 | 2.96 (1.93–4.54) |
| Intercept | −3.86 | < 0.001 | – |
Hosmer–Lemeshow goodness-of-fit χ2 = 13.1, 7 degrees of freedom, p = 0.1.
Area under receiver operating characteristic curve (C statistic) = 0.88 (95% CI 0.85–0.90).
Optimism 0.015.
Optimism-corrected C statistic = 0.87 (95% CI 0.84–0.89).
Note: BP = blood pressure, CI = confidence interval, ECG = electrocardiogram.
Shrinkage-corrected regression coefficients and odds ratios.
Triggered by being in a warm crowded place, prolonged standing, fear, emotion or pain.
Includes coronary or valvular heart disease, cardiomyopathy, congestive heart failure and non-sinus rhythm (ECG evidence during index visit or documented history of ventricular or atrial arrhythmias, or device implantation).
Includes BP readings from triage until disposition from emergency department.
After multiplying the regression coefficients by the shrinkage factor, we created the Canadian Syncope Risk Score (Figure 2). The scores range from −3 to 11, with a shrinkage-adjusted expected risk ranging from 0.4% to 83.6%, respectively. We found that the model was well calibrated, with close agreement between the observed and expected probabilities of a serious adverse event at various score levels (Figure 3). In addition, the Hosmer–Lemeshow goodness-of-fit statistic was not significant (χ2 = 13.1, 7 degrees of freedom, p = 0.11), which confirmed the adequacy of calibration.
Figure 2:
Canadian Syncope Risk Score to identify patients with syncope at risk of serious adverse events within 30 days after disposition from the emergency department. *Triggered by being in a warm crowded place, prolonged standing, fear, emotion or pain. †Includes coronary or valvular heart disease, cardiomyopathy, congestive heart failure and non-sinus rhythm (electrocardiogram evidence during index visit or documented history of ventricular or atrial arrhythmias, or device implantation). ‡Includes blood pressure values from triage until disposition from the emergency department. §Shrinkage-adjusted expected risk.
Figure 3:
Observed and expected probability of serious adverse events among patients with syncope within 30 days after disposition from the emergency department, by risk score determined using the Canadian Syncope Risk Score model. Error bars = 95% confidence intervals for the observed probabilities. Scores of 6 or higher were combined owing to low frequencies.
The distribution of patients with each score and the diagnostic characteristics at specific thresholds are presented in Appendix 4 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151469/-/DC1). The sensitivity was 99.2% (95% CI 95.9%–100%) for a threshold score of −2 or higher and 97.7% (95% CI 93.5%–99.5%) for a threshold score of −1 or higher. Because of small numbers of patients with higher scores, both for calibration and for reporting distribution of patients with each score, we collapsed scores of 6 or higher. When we examined the frequency of predictor selections in the backward elimination across the bootstrap samples, we found that the components of the score were the top 9 predictors selected in the 500 replications (Appendix 5, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151469/-/DC1). We found that the model performed very well in its ability to discriminate between the 4 subtypes of serious adverse event (death, arrhythmia, nonarrhythmic cardiac and noncardiac) as shown in Appendix 6 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151469/-/DC1.
Interpretation
Using data from our prospective multicentre study, we developed the Canadian Syncope Risk Score as a clinical decision tool to identify adult patients at risk of serious adverse events not evident during the initial evaluation of syncope in the emergency department. An important number of patients with syncope had serious adverse events, including death, within 30 days after disposition from the emergency department, with a substantial proportion of events occurring outside the hospital. A score of −2 or lower confers a very low risk (< 1%), scores of −1 to 3 confer a low to medium risk (1%–8%), and scores of 4 or more confer a high or very high risk (> 12%). This scoring system will allow physicians to identify patients with syncope at higher risk of serious outcomes, and after its validation, the tool will aid in the identification and quick disposition of lower-risk patients from the emergency department.
Five prospective syncope studies attempted to develop tools for identification of short-term risk of serious adverse events.8,9,13,14,26 The San Francisco Syncope Rule was developed to predict such outcomes within 7 days.13 It performed poorly on external validation at various study sites, which made it unacceptable for adoption into practice.7,12,27 The Risk Stratification of Syncope in the Emergency Department (ROSE) rule was developed to predict serious adverse events within 1 month.9 The rule performed suboptimally when validated and requires B-type natriuretic peptide testing, which is not easily available in many emergency departments. The Short-Term Prognosis in Syncope (STePS) Study identified risk factors for death and therapeutic interventions within 10 days.14 The STePS predictors “concomitant trauma” and “male sex” make it impractical to use. The Boston Syncope Criteria comprise a long list of 25 plausible predictors for 30-day serious adverse events without mathematical modelling.8 Kayayurt and colleagues26 published the Anatolian Syncope Rule, but it was developed from a small sample of 231 Turkish patients with 39 serious adverse events. Three of the studies13,14,26 included readmission to hospital without identification of serious underlying conditions as an outcome. All of the studies except 214,26 included cortical stroke as an outcome and patients with obvious serious adverse events identified in the emergency department for derivation of the prediction tool. Because cortical strokes are not related to global hypoperfusion, the European Society of Cardiology recommended that they not be included as a serious adverse event.4 Inclusion of patients with serious adverse events identified in the emergency department introduces bias in the tool (e.g., low hematocrit is identified as a risk factor if patients with obvious gastrointestinal bleeding in the emergency department were included, but it is no longer significant after exclusion of these patients).9,13,28
In our study, we found predisposition to vasovagal syncope to be negatively associated with serious adverse events. The Evaluation of Guidelines in Syncope Study (EGSYS) score, developed to identify patients with cardiac syncope, reported a similar finding.29 Similar to our study, a retrospective study to develop a score to predict 30-day serious adverse events among older (≥ 60 yr) patients with syncope found abnormal troponin (> 99th percentile of normal population) an independent risk factor.28 Several previous prospective studies have reported heart disease and abnormal ECG findings as independent risk factors for serious outcomes among patients with syncope.13,14,29 Our final model includes 2 predictors related to the final emergency department diagnosis: vasovagal and cardiac syncope. Although subjective, these predictors had good interobserver agreement and were robust, as evidenced by the odds ratio estimates. In addition, both predictors were selected 100% of the time during the bootstrap internal validation. Use of physicians’ diagnostic impression has been previously incorporated into successful models for venous thromboembolism detection.30
Strengths and limitations
Our study is the largest prospective syncope study to date, with 4030 enrolled patients, including a large number who had serious outcomes after disposition from the emergency department. Previously published syncope tools were developed with fewer than 700 patients. We conducted our study as per methodologic standards for clinical decision tool studies,31 and our report meets all of the criteria listed in the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement.32 We designed our study by avoiding the limitations identified in previous ones, for example by developing a tool for syncope-specific serious outcomes and for serious outcomes after disposition from the emergency department. Different definitions of “abnormal ECG” were used in the previous studies, and none were based on evidence. In our study, we collected ECG predictor variables separately rather than using an a priori definition of “abnormal ECG.” We used robust statistical techniques to develop the score, including safeguards to prevent overfitting and overestimation of the model’s performance. Our score includes relatively simple and well-defined predictors based on clinical evaluation, ECG findings and readily available blood test results that can be easily incorporated into practice.
Our study has limitations. About one-fifth of eligible patients were not enrolled because the emergency physicians were too busy to complete the study forms. However, this is an overestimation because doubtful cases were assigned as eligible non-enrolled patients. We are not aware of systematic reasons for nonrecruitment.
Although it is difficult to establish a clear link between the adverse event and the index syncope, 2 international panels of syncope researchers have recommended that serious conditions occurring in patients with syncope after disposition from the emergency department be reported as outcomes in emergency department risk-stratification studies.15,20
For patients without troponin measurements, we assumed the levels to be normal because the patients were younger with few comorbidities. On the basis of clinical judgment, we believe that troponin does not necessarily need to be measured in all patients with syncope.
Most of the patients with missing predictors were in the group with no serious adverse events. Apart from the troponin predictor, history of heart disease was missing for 6.3% of patients, and the rest were missing for less than 2% of patients. We imputed these missing predictors using multiple imputation. In addition, given the large number of patients with no serious adverse event, we believe the effect of these missing predictors would have been minimal to change the results of our study.
A small proportion of patients were lost to follow-up and not included in our analysis. It is unlikely that many of these patients would have had a serious outcome, and their exclusion is unlikely to have biased our results.
Conclusion
About 1 in 20 patients with syncope experienced severe adverse events, including death, after initial evaluation in the emergency department. Using data from the clinical evaluation, ECG findings and readily available blood test results, and robust methodology, we developed the Canadian Syncope Risk Score to predict which patients will have severe adverse events after disposition from the emergency department. Once validated, the tool will aid in the identification and quick disposition of lower-risk patients from the emergency department. Accurate risk stratification and improved decision-making on patient disposition will benefit both patients and the health care system during the management of acute syncope in the emergency department.
Acknowledgements
The authors gratefully acknowledge the emergency physicians at The Ottawa Hospital (Civic and General campuses), the Kingston General Hospital, the Hotel Dieu Hospital, the Foothills Medical Centre and the University of Alberta Hospital who recruited the patients, and the emergency medicine residents who helped in this process. The authors also acknowledge the following members of their research team: Ottawa site: Pam Ladouceur, Sarah Gaudet, Karen Pratt, Aparna Vaidyanathan, Soo-Min Kim, My-Linh Tran, Sheryl Domingo, Catherine Clement and Angela Marcantonio; Kingston site: Jane Reid, Laura Goodfellow; Edmonton site: Justin Lowes, Pamela Pang and Natalie Runham.
Footnotes
Competing interests: None declared.
Disclaimer: George Wells is a biostatistical consultant for CMAJ and was not involved in the editorial decision-making process for this article.
This article has been peer reviewed.
Contributions: Venkatesh Thiruganasambandamoorthy, Ian Stiell and George Wells conceived the study. All of the authors contributed to the study design and the application for research funding. Venkatesh Thiruganasambandamoorthy, Marco Sivilotti, Brian Rowe, Muhammad Mukarram and Eddy Lang supervised the conduct of the trial and data collection, undertook recruitment of patients and managed the data, including quality control. Monica Taljaard, Jeffrey Perry, Robert Sheldon and George Wells provided statistical advice on the study design. Kenneth Kwong and Monica Taljaard analyzed the data. Venkatesh Thiruganasambandamoorthy drafted the manuscript. All of the authors reviewed the manuscript and contributed substantially to its revision, approved the final version to be published and agreed to act as guarantors of the work.
Funding: The study was funded by the Physicians’ Services Incorporated Foundation (grant no. 09q4017), the Canadian Institutes of Health Research (CIHR grant no. FRN114927) and the Ontario Innovation Fund. During part of the study period, Venkatesh Thiruganasambandamoorthy was supported by the Heart and Stroke Foundation of Canada through the Jump Start Resuscitation Scholarship. Ian Stiell holds the Research Chair from the University of Ottawa. Brian Rowe is supported by the CIHR as a Tier I Canada Research Chair in Evidence-based Emergency Medicine.
References
- 1.Blanc JJ, L’Her C, Touiza A, et al. Prospective evaluation and outcome of patients admitted for syncope over a 1 year period. Eur Heart J 2002;23:815–20. [DOI] [PubMed] [Google Scholar]
- 2.Sun BC, Emond JA, Camargo CA., Jr Characteristics and admission patterns of patients presenting with syncope to US emergency departments, 1992–2000. Acad Emerg Med 2004;11:1029–34. [DOI] [PubMed] [Google Scholar]
- 3.Ganzeboom KS, Mairuhu G, Reitsma JB, et al. Lifetime cumulative incidence of syncope in the general population: a study of 549 Dutch subjects aged 35–60 years. J Cardiovasc Electrophysiol 2006;17:1172–6. [DOI] [PubMed] [Google Scholar]
- 4.Moya A, Sutton R, Ammirati F, et al. Guidelines for the diagnosis and management of syncope (version 2009). Eur Heart J 2009;30:2631–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kapoor WN. Evaluation and outcome of patients with syncope. Medicine 1990;69:160–75. [DOI] [PubMed] [Google Scholar]
- 6.Quinn J, McDermott D, Stiell I, et al. Prospective validation of the San Francisco syncope rule to predict patients with serious outcomes. Ann Emerg Med 2006;47:448–54. [DOI] [PubMed] [Google Scholar]
- 7.Thiruganasambandamoorthy V, Hess EP, Alreesi A, et al. External validation of the San Francisco syncope rule in the Canadian setting. Ann Emerg Med 2010;55:464–72. [DOI] [PubMed] [Google Scholar]
- 8.Grossman SA, Fischer C, Lipsitz LA, et al. Predicting adverse outcomes in syncope. J Emerg Med 2007;33:233–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Reed MJ, Newby DE, Coull AJ, et al. The ROSE (risk stratification of syncope in the emergency department) study. J Am Coll Cardiol 2010;55:713–21. [DOI] [PubMed] [Google Scholar]
- 10.Pires LA, Ganji JR, Jarandila R, et al. Diagnostic patterns and temporal trends in the evaluation of adult patients hospitalized with syncope. Arch Intern Med 2001;161:1889–95. [DOI] [PubMed] [Google Scholar]
- 11.Thiruganasambandamoorthy V, Taljaard M, Stiell IG, et al. Emergency department management of syncope: need for standardization and improved risk stratification. Intern Emerg Med 2015;10:619–27. [DOI] [PubMed] [Google Scholar]
- 12.Serrano LA, Hess EP, Bellolio MF, et al. Accuracy and quality of clinical decision rules for syncope in the emergency department: a systematic review and meta-analysis. Ann Emerg Med 2010;56:362–373.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Quinn JV, Stiell IG, McDermott DA, et al. Derivation of the San Francisco syncope rule to predict patients with short-term serious outcomes. Ann Emerg Med 2004;43:224–32. [DOI] [PubMed] [Google Scholar]
- 14.Costantino G, Perego F, Dipaola F, et al. Short- and long-term prognosis of syncope, risk factors, and role of hospital admission: results from the STePS (Short-Term Prognosis of Syncope) study. J Am Coll Cardiol 2008;51:276–83. [DOI] [PubMed] [Google Scholar]
- 15.Sun BC, Thiruganasambandamoorthy V, Cruz JD. Consortium to Standardize ED Syncope Risk Stratification Reporting. Standardized reporting guidelines for emergency department syncope risk-stratification research. Acad Emerg Med 2012;19:694–702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Thiruganasambandamoorthy V, Kwong K, Stiell IG, et al. Short-term risk of arrhythmias among emergency department syncope patients with non-sinus rhythm. Int J Cardiol 2015;189:12–4. [DOI] [PubMed] [Google Scholar]
- 17.Sheldon RS, Morillo CA, Krahn AD, et al. Standardized approaches to the investigation of syncope: Canadian Cardiovascular Society position paper. Can J Cardiol 2011;27:246–53. [DOI] [PubMed] [Google Scholar]
- 18.Thiruganasambandamoorthy V, Wells GA, Hess EP, et al. Derivation of a risk scale and quantification of risk factors for serious adverse events in adult emergency department syncope patients. CJEM 2014;16:120–30. [DOI] [PubMed] [Google Scholar]
- 19.Thiruganasambandamoorthy V, Stiell IG, Sivilotti ML, et al. Risk stratification of adult emergency department syncope patients to predict short-term serious outcomes after discharge (RiSEDS) study. BMC Emerg Med 2014;14:8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Sun BC, Costantino G, Barbic F, et al. Priorities for emergency department syncope research. Ann Emerg Med 2014;64:649–55.e2. [DOI] [PubMed] [Google Scholar]
- 21.Steyerberg EW. Clinical prediction models: a practical approach to development, validation and updating. New York: Springer; 2009. [Google Scholar]
- 22.Little RJ, Rubin DB. Statistical analysis with missing data. 2nd ed New York: John Wiley and Sons; 2002. [Google Scholar]
- 23.Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc 1983;78:316–31. [Google Scholar]
- 24.Le Gal G, Righini M, Roy PM, et al. Prediction of pulmonary embolism in the emergency department: the revised Geneva score. Ann Intern Med 2006;144:165–71. [DOI] [PubMed] [Google Scholar]
- 25.Scheaffer RL, Mendenhall W, Ott L. Elementary survey sampling. 2nd ed Pacific Grove (CA): Duxbury Press; 1979. [Google Scholar]
- 26.Kayayurt K, Akoglu H, Limon O, et al. Comparison of existing syncope rules and newly proposed Anatolian syncope rule to predict short-term serious outcomes after syncope in the Turkish population. Int J Emerg Med 2012;5:17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Reed MJ, Newby DE, Coull AJ, et al. The Risk stratification Of Syncope in the Emergency department (ROSE) pilot study: a comparison of existing syncope guidelines. Emerg Med J 2007;24:270–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Sun BC, Derose SF, Liang LJ, et al. Predictors of 30-day serious events in older patients with syncope. Ann Emerg Med 2009;54: 788.e1–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Del Rosso A, Ungar A, Maggi R, et al. Clinical predictors of cardiac syncope at initial evaluation in patients referred urgently to a general hospital: the EGSYS score. Heart 2008;94:1620–6. [DOI] [PubMed] [Google Scholar]
- 30.Wells P, Anderson D. The diagnosis and treatment of venous thromboembolism [abstract]. Hematology Am Soc Hematol Educ Program 2013;2013:457–63. [DOI] [PubMed] [Google Scholar]
- 31.Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999;33:437–47. [DOI] [PubMed] [Google Scholar]
- 32.Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162:55–63. [DOI] [PubMed] [Google Scholar]