Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 May 5.
Published in final edited form as: Arch Intern Med. 2008 May 12;168(9):959–968. doi: 10.1001/archinte.168.9.959

Factors Associated With Longer Time From Symptom Onset to Hospital Presentation for Patients With ST-Elevation Myocardial Infarction

Henry H Ting 1, Elizabeth H Bradley 1, Yongfei Wang 1, Judith H Lichtman 1, Brahmajee K Nallamothu 1, Mark D Sullivan 1, Bernard J Gersh 1, Veronique L Roger 1, Jeptha P Curtis 1, Harlan M Krumholz 1
PMCID: PMC4858313  NIHMSID: NIHMS686507  PMID: 18474760

Abstract

Background

Previous studies have demonstrated the effects of single factors, such as age, sex, and race, with longer delays from symptom onset to hospital presentation in patients with ST-elevation myocardial infarction.

Methods

We studied risk factors individually and in combination to determine the cumulative effect on delay times in 482 327 patients with ST-elevation myocardial infarction enrolled in the National Registry of Myocardial Infarction between January 1, 1995, and December 31, 2004. We analyzed patient subgroups with the following risk factors in combination: younger than 70 years vs 70 years and older, race/ethnicity, men vs women, and nondiabetic vs diabetic.

Results

The geometric mean for delay time was 114 minutes, with a decreasing trend from 123 minutes in 1995 to 113 minutes in 2004 (P<.001). Nearly half of the patients (45.5%) presented more than 2 hours and 8.7% presented more than 12 hours after the onset of symptoms. Compared with the reference group (those <70 years, men, white, and did not have diabetes mellitus [DM]), subgroups with longer delay times (P<.01 for all) included those younger than 70 years, men, black, and had DM (+43 minutes); those younger than 70 years, women, black, and had DM (+55 minutes); those 70 years and older, men, black, and had DM (+60 minutes); and those 70 years and older, women, black, and had DM (+63 minutes).

Conclusions

Patient subgroups with a combination of factors (older age, women, Hispanic or black race, and DM) have particularly long delay times that may be 60 minutes longer than subgroups without those characteristics. Improving patient responsiveness in these subgroups represents an important opportunity to improve quality of care and minimize disparities in care.


Time from symptom onset to presentation to the hospital is particularly important for patients with ST-elevation myocardial infarction (STEMI). Aside from the association of longer ischemic times with more myocardial damage and adverse clinical consequences,18 the effectiveness of reperfusion therapy depends on this interval, with the therapy having greatest benefit for patients who present with the shortest delay.911 Previous studies1217 have demonstrated that delays to hospital presentation average 2 hours and are more commonly seen in elderly patients, black patients, women, patients with diabetes mellitus (DM), and those with atypical symptoms. Although previous studies have focused on the effect of single factors associated with delays in hospital presentation, to our knowledge, the cumulative effect of having multiple demographic and clinical risk factors associated with longer delays has not been shown. Furthermore, little information about delay is available from contemporary and nationally representative data from patients with STEMI.

Understanding variations and meaningful differences in delay in patient subgroups may help with the design of interventions to improve patient responsiveness and access. Moreover, interventions should target subgroups at greatest risk for delay, in addition to mass media campaigns directed toward an entire population or community.17 In particular, delays in hospital presentation may be concentrated in vulnerable subgroups in the population and could contribute to disparities in health care. To address these questions, we undertook a study to evaluate patient and hospital factors associated with longer delays in hospital presentation; patient sub-groups at highest risk for delay as defined by age, race/ ethnicity, sex, and clinical characteristics; and trends in delay in patients with STEMI between 1995 and 2004 from the National Registry of Myocardial Infarction (NRMI).

METHODS

STUDY DESIGN AND SAMPLE

The study sample included patients enrolled in the NRMI, a voluntary prospective registry of patients with acute myocardial infarction (AMI) between January 1, 1995, and December 31, 2004. Participating hospitals, data collection methods, verification methods, and reliability have been previously described.18,19 The NRMI criteria for the diagnosis of AMI used the International Classification of Diseases, Ninth Revision, Clinical Modification, discharge diagnosis code of 410.X1 and the diagnosis was confirmed with 1 of the following criteria: a 2-fold or greater elevation of cardiac biomarkers, electrocardiographic (ECG) evidence, and echocardiographic, scintigraphic, or autopsy evidence. Participating hospitals, if required, obtained institutional review board approval for NRMI data abstraction.

Between 1995 and 2004, there were 1 926 108 admissions for AMI in the NRMI. The following patients were excluded sequentially: those who did not have new or presumed new ST-segment elevation in 2 or more leads or left bundle branch block on the first ECG (n=1 161 187), those who developed symptoms of AMI after hospital admission (n=14 433), those who had an unknown time of symptom onset (n=173 051), those who had a first ECG time that was not the diagnostic ECG time for STEMI (n=71 842), and those who had an unknown time of first ECG (n=23 268). The remaining 482 327 patients with STEMI composed the study population for the analysis of variables associated with delay from symptom onset to hospital presentation.

DATA COLLECTION AND MEASURES

Delay in hospital presentation was calculated from the documented date and time of symptom onset to the documented date and time of hospital arrival. For the outcome of delay in hospital presentation, we log transformed the outcome measure and performed parametric analysis because the distribution was skewed. To improve the clinical interpretability of the results, we converted the logged values from the models back to their original units (ie, minutes) using geometric means20,21 and simulation with 10 000 reiterations.22 The geometric mean gives less weight to outlying values and, thus, better reflects the median compared with the arithmetic mean.

For the candidate factors associated with delay in hospital presentation, we considered patient and hospital variables. Patient variables included age; sex; race/ethnicity (abstracted from medical records and categorized as white, black, Hispanic, Asian, and other or unknown); payer type (categorized as commercial insurance, Medicare only, Medicare and any other insurance, Medicaid or self-pay, and other or unknown); medical history (current smoker, DM, hypertension, hypercholesterolemia, family history of coronary artery disease, previous MI, previous congestive heart failure [CHF], previous percutaneous coronary intervention [PCI], previous coronary artery bypass graft [CABG] surgery, previous stroke, previous angina, absence of chest pain at presentation, CHF at presentation, cardiogenic shock at presentation, systolic blood pressure <90 mm Hg at presentation, and heart rate > 100 beats/min at presentation); and time of day and day of week at presentation (weekdays were defined as Monday to Friday and included daytime [8 AM to 4 PM], evening [4 PM to midnight], and night [midnight to 8 AM]; weekends were defined as Saturday and Sunday and included daytime [8 AM to 4 PM], evening [4 PM to midnight], and night [midnight to 8 AM]). Hospital variables included US Census region (West, South, Midwest, and Northeast), teaching hospitals (defined as participation in an accredited residency or fellowship training program), and type of cardiac facilities (interventional, interventional without surgery on site, invasive but not interventional, and noninvasive). All these variables were selected based on their clinical and statistical significance from previous studies.8,23,24

STATISTICAL ANALYSES

We plotted the distribution of the interval from symptom onset to hospital presentation and performed tests of the linear trend from calendar year 1995 to 2004 of geometric means of those intervals. We performed bivariate and multivariate generalized linear models to estimate the associations between candidate factors and delay in hospital presentation. Factors associated with delay in hospital presentation were identified using the generalized linear model with the stepwise selection method (entry significance level of P<.10), and then significant factors were chosen by a significance level of P<.05 and defining clinically meaningful delay time as greater than 5 minutes compared with the respective reference group.

We analyzed patient subgroups at highest risk for longer delays by examining the following risk factors alone and in combination: younger than 70 years vs 70 years and older, race (white, black, Hispanic, and other), men vs women, and non-diabetic vs diabetic. We also evaluated trends in delay in hospital presentation between 1995 and 2004 for patients with particular demographic and clinical characteristics at risk for delay. We constructed the test of overall differences and linear trend in delay in hospital presentation for each group.

All the previous analyses were repeated after excluding patients who were transferred in from another hospital, and also in the 1995 to 2004 cohort of patients for whom unique hospital identifiers were available and hierarchical linear models could be applied to account for the clustering of patients within hospitals. These results were not reported separately because the direction and magnitude of the effects were similar to those of the previous analyses and did not change the conclusions. Statistical analyses were performed using SAS version 9.1 (SAS Institute Inc, Cary, North Carolina) and Stata version 8.0 (Stata Corp, College Station, Texas).

RESULTS

STUDY POPULATION

The patient and hospital characteristics of the study population are given in Table 1; most patients were younger than 70 years (64.5%), male (66.8%), and white (86.2%). Payer types included commercial insurance (39.2%), Medicare only (29.2%), Medicare with any other insurance (13.2%), and Medicaid or self-pay (10.8%). Clinical characteristics included patients with DM (21.5%), previous MI (19.0%), previous PCI (9.6%), previous CHF (6.8%), and absence of chest pain at presentation (9.8%). Nearly half of the patients (46.4%) presented during daytime hours (8 AM to 4 PM) on either weekdays or weekends.

Table 1.

Baseline Patient and Hospital Characteristics and Delay in Hospital Presentation

Description Patients, No. (%) (N=482 327) Estimate of Delay, Geometric Mean (95% CI), mina Difference in Delay, Geometric Mean (95% CI), mina P Value
Age, y <.001
 <60 196 862 (40.8) 107.4 (106.8 to 107.9) 0 [Reference]
 60–69 114 271 (23.7) 121.6 (120.8 to 122.4) 14.2 (13.2 to 15.2) <.001
 70–79 106 787 (22.1) 138.8 (137.9 to 139.8) 31.5 (30.4 to 32.6) <.001
 ≥80 64 407 (13.4) 155.2 (153.8 to 156.6) 47.8 (46.3 to 49.3) <.001
Female sex
 No 322 142 (66.8) 114.5 (114.0 to 114.9) 0 [Reference]
 Yes 160 185 (33.2) 142.0 (141.2 to 142.8) 27.5 (26.6 to 28.4) <.001
Race <.001
 White 415 912 (86.2) 121.7 (121.3 to 122.1) 0 [Reference]
 Black 24 646 (5.1) 141.0 (139.0 to 143.0) 19.3 (17.2 to 21.4) <.001
 Hispanic 13 972 (2.9) 135.8 (133.3 to 138.5) 14.1 (11.5 to 16.8) <.001
 Asian 6978 (1.4) 124.6 (121.2 to 128.0) 2.9 (−0.5 to 6.3) .10
 Other or unknown 20 819 (4.3) 119.8 (117.9 to 121.7) −1.9 (−3.8 to 0.1) .06
Health insurance <.001
 Commercial (HMO/PPO) only 188 991 (39.2) 109.8 (109.2 to 110.4) 0 [Reference]
 Medicare only 141 063 (29.2) 140.4 (139.5 to 141.2) 30.6 (29.5 to 31.6) <.001
 Medicare with any other insurance 63 472 (13.2) 134.8 (133.6 to 136.0) 24.9 (23.6 to 26.3) <.001
 Medicaid or self-pay 51 926 (10.8) 118.6 (117.5 to 119.8) 8.8 (7.5 to 10.1) <.001
 Other or unknown 36 875 (7.6) 118.8 (117.4 to 120.2) 9.0 (7.5 to 10.5) <.001
Current smoker
 No 307 630 (63.8) 127.7 (127.2 to 128.2) 0 [Reference]
 Yes 174 697 (36.2) 115.1 (114.5 to 115.7) −12.6 (−13.4 to −11.8) <.001
Diabetes mellitus
 No 378 842 (78.5) 117.0 (116.5 to 117.4) 0 [Reference]
 Yes 103 485 (21.5) 147.6 (146.6 to 148.6) 30.6 (29.5 to 31.7) <.001
Previous MI
 No 390 925 (81.0) 124.8 (124.3 to 125.2) 0 [Reference]
 Yes 91 402 (19.0) 115.4 (114.6 to 116.3) −9.3 (−10.3 to −8.4) <.001
Hypertension
 No 246 430 (51.1) 115.8 (115.3 to 116.3) 0 [Reference]
 Yes 235 897 (48.9) 130.9 (130.3 to 131.5) 15.1 (14.3 to 15.9) <.001
Hypercholesterolemia
 No 329 180 (68.2) 125.2 (124.7 to 125.7) 0 [Reference]
 Yes 153 147 (31.8) 118.3 (117.6 to 118.9) −7.0 (−7.8 to −6.1) <.001
Family history of CAD
 No 333 202 (69.1) 124.2 (123.7 to 124.7) 0 [Reference]
 Yes 149 125 (30.9) 120.2 (119.5 to 120.9) −4.0 ( 4.8 to −3.1) <.001
Previous CHF
 No 449 371 (93.2) 121.2 (120.8 to 121.6) 0 [Reference]
 Yes 32 956 (6.8) 149.5 (147.6 to 151.3) 28.3 (26.4 to 30.2) <.001
Previous PCI
 No 435 955 (90.4) 125.7 (125.3 to 126.2) 0 [Reference]
 Yes 46 372 (9.6) 99.7 (98.7 to 100.8) −26.0 ( 27.1 to −24.9) <.001
Previous CABG
 No 445 642 (92.4) 122.9 (122.5 to 123.4) 0 [Reference]
 Yes 36 685 (7.6) 123.1 (121.7 to 124.6) 0.2 (−1.3 to 1.7) .80
Previous stroke
 No 454 251 (94.2) 122.0 (121.6 to 122.4) 0 [Reference]
 Yes 28 076 (5.8) 140.1 (138.2 to 142.0) 18.1 (16.2 to 20.1) <.001
Previous angina
 No 431 863 (89.5) 122.5 (122.0 to 122.9) 0 [Reference]
 Yes 50 464 (10.5) 127.2 (126.0 to 128.5) 4.8 (3.4 to 6.1) <.001
Chest pain
 Yes 434 997 (90.2) 119.5 (119.1 to 119.9) 0 [Reference]
 No 47 330 (9.8) 159.7 (158.0 to 161.3) 40.2 (38.5 to 41.9) <.001
Cardiogenic shock
 No 473 142 (98.1) 123.6 (123.2 to 124.0) 0 [Reference]
 Yes 9185 (1.9) 94.5 (92.2 to 96.7) −29.1 (−31.4 to −26.9) <.001
Systolic blood pressure <90 mm Hg
 No 458 930 (95.1) 124.7 (124.2 to 125.1) 0 [Reference]
 Yes 23 397 (4.9) 93.8 (92.4 to 95.2) −30.8 (−32.3 to −29.4) <.001
Pulse >100 beats/min
 No 407 345 (84.5) 118.3 (117.9 to 118.7) 0 [Reference]
 Yes 74 982 (15.5) 151.5 (150.3 to 152.8) 33.2 (31.9 to 34.5) <.001
Current CHF
 No 414 204 (85.9) 118.8 (118.4 to 119.2) 0 [Reference]
 Yes 68 123 (14.1) 151.8 (150.5 to 153.1) 33.0 (31.6 to 34.4) <.001
Time of presentation <.001
 Weekday daytime 158 925 (32.9) 131.8 (131.0 to 132.5) 0 [Reference]
 Weekday evening 94 066 (19.5) 116.7 (115.9 to 117.6) −15.1 (−16.2 to −13.9) <.001
 Weekday night 86 687 (18.0) 119.6 (118.7 to 120.5) −12.2 (−13.4 to −11.0) <.001
 Weekend daytime 65 355 (13.5) 122.3 (121.2 to 123.4) − 9.5 (−10.8 to −8.2) <.001
 Weekend evening 40 869 (8.5) 113.6 (112.3 to 114.9) − 18.2 (−19.7 to −16.7) <.001
 Weekend night 36 425 (7.6) 122.7 (121.2 to 124.1) − 9.1 (−10.8 to −7.5) <.001
Year of presentation <.001
 1995 63 153 (13.1) 128.8 (127.7 to 130.0) 0 [Reference]
 1996 65 650 (13.6) 126.6 (125.5 to 127.7) − 2.2 (− 3.8 to − 0.6) .007
 1997 62 390 (12.9) 124.1 (123.0 to 125.2) −4.7 (−6.3 to −3.1) <.001
 1998 59 685 (12.4) 126.4 (125.3 to 127.6) −2.4 (−4.0 to −0.7) .004
 1999 64 679 (13.4) 124.5 (123.4 to 125.6) −4.4 (−6.0 to −2.8) <.001
 2000 48 576 (10.1) 120.3 (119.1 to 121.6) −8.5 (−10.2 to −6.8) <.001
 2001 44 043 (9.1) 117.7 (116.4 to 119.0) −11.1 (−12.9 to −9.4) <.001
 2002 29 567 (6.1) 117.6 (116.0 to 119.2) −11.2 (−13.2 to −9.3) <.001
 2003 25 350 (5.3) 113.0 (111.4 to 114.6) −15.8 (−17.8 to −13.8) <.001
 2004 19 234 (4.0) 114.1 (112.3 to 116.0) −14.7 (−16.9 to −12.5) <.001
Cardiac facilities <.001
 Noninvasive 72 518 (15.0) 123.9 (122.9 to 125.0) 0 [Reference]
 Invasive but noninterventional 94 031 (19.5) 128.4 (127.5 to 129.4) 4.5 (3.1 to 5.9) <.001
 Interventional 291 634 (60.5) 120.9 (120.4 to 121.4) −3.1 (−4.2 to −1.9) <.001
 Interventional without on-site surgery 24 144 (5.0) 124.6 (122.9 to 126.5) 0.7 (−1.4 to 2.8) .50
Teaching status
 No 275 005 (57.0) 121.8 (121.2 to 122.3) 0 [Reference]
 Yes 207 322 (43.0) 124.6 (124.0 to 125.2) 2.8 (2.0 to 3.6) < .001
Census division < .001
 West 116 712 (24.2) 122.2 (121.4 to 123.0) 0 [Reference]
 South 150 716 (31.2) 123.0 (122.3 to 123.7) 0.8 (−0.3 to 1.9) .10
 Midwest 144 405 (29.9) 117.8 (117.1 to 118.5) −4.4 (−5.5 to −3.4) < .001
 Northeast 70 494 (14.6) 135.6 (134.5 to 136.8) 13.4 (12.0 to 14.9) < .001

Abbreviations: CABG, coronary artery bypass graft; CAD, coronary artery disease; CHF, congestive heart failure; CI, confidence interval; HMO, health maintenance organization; MI, myocardial infarction; PCI, percutaneous coronary intervention; PPO, preferred provider organization.

a

Log transformation was performed on dependent variables in the model, and simulation was performed to convert the results back to the natural units.

The geometric mean for delay time was 114 minutes during the study period, with a trend toward shorter times (from 123 minutes in 1995 to 113 minutes in 2004, P<.001). Figure 1 shows the patient distribution as a function of delay times, and nearly half of the patients (45.5%) presented more than 2 hours after the onset of symptoms, 8.0% presented 6 hours to 12 hours after the onset of symptoms, and 8.7% presented more than 12 hours after the onset of symptoms. Using the cutoff value of 12 hours for delay in hospital presentation as the eligibility window for reperfusion therapy as defined by the guidelines,24 the proportion of patients presenting more than 12 hours after the onset of symptoms for the following groups was as follows: younger than 60 years, 7.1%; 60 to 69 years, 8.3%; 70 to 79 years, 10.1%; 80 years and older, 12.0%; men, 7.8%; women, 10.5%; white patients, 8.5%; black patients, 10.9%; Hispanic patients, 11.1%; patients with commercial insurance, 7.3%; patients with Medicare only, 10.4%; patients with Medicare and any other insurance, 9.6%; and patients with Medicaid or patients who self-pay, 8.3%.

Figure 1.

Figure 1

Distribution of time from symptom onset to hospital presentation.

MULTIVARIATE ANALYSIS OF FACTORS ASSOCIATED WITH DELAY

Patient and hospital characteristics associated with longer time from symptom onset to hospital presentation are given in Table 2. Compared with patients younger than 60 years, adjusted delay in hospital presentation was longer by 9, 19, and 29 minutes for patients aged 60 to 69 years, 70 to 79 years, and 80 years and older, respectively (P<.001 for all). Compared with respective reference groups, adjusted time between symptom onset and hospital presentation was longer by 12 minutes for women, 14 minutes for patients identified as black, 11 minutes for patients identified as Hispanic, 18 minutes for patients with DM, and 18 minutes for patients without chest pain at presentation (P<.001 for all). Conversely, time from symptom onset to hospital presentation was shorter in patients who had previous MI (−8 minutes), who had previous PCI (−16 minutes), or who had signs of shock (−23 minutes) or hypotension (−27 minutes) at presentation (P<.001 for all). Compared with weekday daytime (8 AM to 4 PM), patients who presented during any other time of day or day of week had shorter delays, with the largest magnitude seen during weekday evenings (4 PM to midnight) (−12 minutes), weekday nights (midnight to 8 AM) (−9 minutes), weekend evenings (4 PM to midnight) (−14 minutes), and weekend nights (midnight to 8 AM) (− 7 minutes) (P<.001 for all). Compared with patients from the West census region, those from the Midwest had shorter delays (−4 minutes) and those from the Northeast had longer delays (+7 minutes) in hospital presentation (P<.001 for both).

Table 2.

Significant Factors in the Multivariate Generalized Linear Model for Delay in Hospital Presentation

Description Model Resultsa
Difference in Delay, Mean (95% CI), minb
Estimate SE t Value P Value
Demographics
Age, y 494.74 <.001
 <60 0 [Reference]
 60–69 0.087 0.0047 18.66 <.001 9.2 (8.2 to 10.1)
 70–79 0.18 0.0059 30.32 <.001 19.3 (18.0 to 20.6)
 ≥80 0.26 0.0070 37.19 <.001 29.3 (27.6 to 31.1)
Female sex
 No 0 [Reference]
 Yes 0.11 0.0037 29.99 <.001 11.5 (10.7 to 12.3)
Race 95.38 <.001
 White 0 [Reference]
 Black 0.13 0.0076 16.99 <.001 14.0 (12.3 to 15.8)
 Hispanic 0.10 0.0099 10.46 <.001 11.2 (9.0 to 13.4)
 Asian 0.024 0.0140 1.75 .08 2.5 (−0.3 to 5.4)
 Other or unknown 0.0094 0.0082 1.16 .20 1.0 (−0.7 to 2.7)
Health insurance 48.65 <.001
 Commercial (HMO/PPO) only 0 [Reference]
 Medicare only 0.058 0.0052 11.27 <.001 6.0 (5.0 to 7.1)
 Medicare with any other insurance 0.038 0.0064 5.97 <.001 4.0 (2.6 to 5.3)
 Medicaid or self-pay 0.059 0.0057 10.36 <.001 6.2 (5.0 to 7.4)
 Other or unknown 0.034 0.0065 5.19 <.001 3.5 (2.2 to 4.9)

Medical History

Diabetes mellitus
 No 0 [Reference]
 Yes 0.17 0.0041 41.31 <.001 18.4 (17.4 to 19.4)
Previous MI
 No 0 [Reference]
 Yes −0.080 0.0047 −17.11 <.001 −8.0 (−8.9 to −7.1)
Hypertension
 No 0 [Reference]
 Yes 0.051 0.0034 14.99 <.001 5.2 (4.6 to 5.9)
Previous PCI
 No 0 [Reference]
 Yes −0.16 0.0061 −27.03 <.001 −15.8 (−16.8 to −14.7)

Presentation

Chest pain
 Yes 0 [Reference]
 No 0.16 0.0058 27.95 <.001 17.6 (16.3 to 19.0)
Cardiogenic shock
 No 0 [Reference]
 Yes −0.26 0.012 −20.91 <.001 −23.5 (−25.4 to −21.5)
Systolic blood pressure <90 mm Hg
 No 0 [Reference]
 Yes −0.29 0.0078 −37.74 <.001 −26.6 (−27.8 to −25.3)
Pulse >100 beats/min
 No 0 [Reference]
 Yes 0.16 0.0047 33.63 <.001 17.2 (16.1 to 18.3)
Current CHF
 No 0 [Reference]
 Yes 0.081 0.0050 16.13 <.001 8.6 (7.5 to 9.7)
Time of presentation 196.76 <.001
 Weekday daytime 0 [Reference]
 Weekday evening −0.12 0.0047 −25.79 <.001 −11.9 (−12.9 to −11.0)
 Weekday night −0.095 0.0048 −19.74 <.001 −9.4 (−10.4 to −8.5)
 Weekend daytime −0.060 0.0053 −11.30 <.001 −6.0 (−7.0 to −5.0)
 Weekend evening −0.14 0.0063 −22.33 <.001 −13.6 (−14.8 to −12.5)
 Weekend night −0.070 0.0066 −10.55 <.001 −6.9 (−8.2 to −5.7)
Year of presentation 28.73 <.001
 1995 0 [Reference]
 1996 −0.018 0.0063 −2.78 .005 −1.8 (−3.1 to −0.5)
 1997 −0.034 0.0064 −5.24 <.001 −3.4 (−4.7 to −2.1)
 1998 −0.028 0.0066 −4.32 <.001 −2.9 (−4.2 to −1.6)
 1999 −0.042 0.0065 −6.42 <.001 −4.2 (−5.5 to −2.9)
 2000 −0.068 0.0070 −9.76 <.001 −6.8 (−8.2 to −5.5)
 2001 −0.082 0.0072 −11.37 <.001 −8.2 (− 9.6 to −6.8)
 2002 −0.074 0.0082 −9.08 <.001 −7.4 (−8.9 to −5.8)
 2003 −0.088 0.0087 −10.09 <.001 −8.7 (−10.4 to −7.0)
 2004 −0.081 0.0096 −8.39 <.001 −8.0 (−9.8 to −6.1)
Hospital Characteristics
Census division 123.55 <.001
 West 0 [Reference]
 South −0.0052 0.0046 −1.14 .30 −0.5 (−1.5 to 0.4)
 Midwest −0.040 0.0046 −8.71 <.001 −4.1 (−5.1 to −3.2)
 Northeast 0.064 0.0058 11.09 <.001 6.7 (5.5 to 7.9)

Abbreviations: See Table 1.

a

The intercept estimate was 4.63 (SE, 0.0083), t value = 559.10, P <.001.

b

Log transformation was performed on dependent variables in the model, and simulation was performed to convert the results back to the natural units.

MULTIVARIATE ANALYSIS OF PATIENT SUBGROUPS WITH MULTIPLE FACTORS

We evaluated 4 variables (age, sex, race/ethnicity, and DM) associated with the greatest risk of longer times, and we examined different combinations to determine the magnitude of effect on adjusted time from symptom onset to hospital presentation. Compared with the reference group of younger (<70 years) white men without DM, we found that the following subgroups had substantially longer times from onset of symptoms to hospital presentation: younger men who were identified as black and had DM (+43 minutes), younger men who were identified as Hispanic and had DM (+47 minutes), younger women who were identified as black and had DM (+55 minutes), younger women who were identified as Hispanic and had DM (+59 minutes), older (≥70 years) men who were identified as black and had DM (+60 minutes), older men who were identified as Hispanic and had DM (+51 minutes), older women who were identified as black and had DM (+63 minutes), and older women who were identified as Hispanic and had DM (+51 minutes) (P<.01 for all) (Figure 2).

Figure 2.

Figure 2

Difference in delay time among patients without (A) and with (B) diabetes mellitus. *The reference group for comparison is white, men, younger than 70 years, and without diabetes mellitus.

MULTIVARIATE ANALYSIS OF TRENDS IN DELAY FOR GROUPS

Between 1995 and 2004, demographic factors that have previously been shown to be associated with longer times from symptom onset to hospital arrival all showed significant improvement across time. In elderly patients (≥70 years), times decreased from 155 to 135 minutes (P=.02), In women, times decreased from 152 to 134 minutes (P=.02). In patients identified as black, Hispanic, Asian, or other nonwhite races/ethnicities, times decreased from 141 to 127 minutes (P=.03). In patients who did not have commercial insurance, times decreased from 140 to 126 minutes (P=.004) (Table 3). Despite these improvements, each of these demographic groups had significantly longer times from symptom onset to hospital presentation than those without these characteristics for every year during the period (P <.001 for all). Among patients who had DM, previous MI, previous PCI, previous CHF, or previous CABG surgery, times also improved from symptom onset to hospital arrival between 1995 and 2004, but these trends did not achieve statistical significance. Among patients with these clinical factors, only those with previous MI or previous PCI had shorter delays than their respective reference groups without these clinical risks, and patients with previous MI or previous PCI also demonstrated minimal improvement in times to hospital presentation between 1995 and 2004.

Table 3.

Adjusted Delay in Hospital Presentation by Calendar Yeara

Description 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 P Value
No Difference Linear Trend Interaction
Demographic Factors
Elderly (aged ≥70 y)
 No 116.0 114.5 112.8 114.3 111.9 110.2 109.6 109.7 108.5 109.7 <.001 .08 <.001
 Yes 154.8 150.7 147.8 146.1 146.1 139.2 135.6 138.1 135.4 135.0 <.001 .02
Female sex
 No 117.9 116.4 115.4 116.5 114.8 111.9 111.4 111.7 109.9 111.3 <.001 .05 <.001
 Yes 152.0 147.9 143.6 143.1 141.5 137.5 133.0 135.7 134.4 133.9 <.001 .02
Nonwhite race
 No 126.5 125.1 122.9 123.4 121.6 118.8 117.1 117.7 116.5 117.1 <.001 .03 .15
 Yes 140.8 132.5 131.9 133.0 132.8 126.4 125.3 128.5 124.2 127.0 <.001 .03
Not commercial (HMO/PPO) insurance only
 No 112.8 111.2 111.1 112.0 109.6 106.9 107.2 107.4 106.6 107.0 <.001 .17 <.001
 Yes 139.7 136.8 133.3 133.6 132.5 128.8 125.6 127.3 125.0 126.3 <.001 .004
Any of these demographic factors
 No 104.0 104.7 103.0 104.9 101.7 100.7 100.5 99.5 99.2 99.3 <.001 .46 .003
 Yes 137.4 134.0 131.8 132.0 130.9 126.7 124.5 126.3 124.1 125.3 <.001 <.001

Clinical Factors
Diabetes mellitus
 No 121.3 119.6 117.5 118.6 116.8 114.5 113.4 114.3 112.2 113.3 <.001 .03 <.001
 Yes 157.5 152.7 151.6 149.9 148.9 141.9 138.0 139.0 139.2 139.2 <.001 .08
Previous MI
 No 130.4 128.5 12.2 126.6 125.0 121.3 119.4 120.8 119.0 119.0 <.001 .002 .14
 Yes 119.5 115.9 115.1 117.1 115.5 114.1 113.6 112.3 111.5 113.4 .006 .52
Previous CHF
 No 126.2 124.3 122.2 123.1 121.3 118.2 116.4 117.5 116.2 116.8 <.001 .001 .11
 Yes 160.1 150.8 151.0 151.2 150.5 146.6 146.7 144.8 135.2 140.5 .004 .66
Previous PCI
 No 131.4 128.8 126.5 127.4 125.8 122.4 120.8 121.7 120.3 120.7 <.001 .002 .69
 Yes 101.8 102.6 103.4 101.8 100.3 98.1 96.3 97.7 94.6 97.7 <.001 .47
Previous CABG
 No 128.6 126.3 124.2 124.6 122.9 119.6 118.1 118.8 117.4 118.4 <.001 .002 .008
 Yes 123.2 121.0 122.1 126.6 126.3 123.2 120.4 124.4 119.6 118.5 .22 .19
Any of these clinical factors
 No 123.1 121.8 119.2 119.7 119.1 115.9 114.8 115.9 114.5 114.3 <.001 .14 .40
 Yes 135.6 132.0 131.1 132.1 129.1 125.7 123.2 123.9 121.9 124.6 <.001 .01

Abbreviations: See Table 1.

a

Log transformation was performed on dependent variables in the model, and simulation was performed to convert the results back to the natural units.

COMMENT

In this study of 482 327 hospital admissions of patients with STEMI, we found that delay from symptom onset to hospital presentation averaged 114 minutes, but some patient subgroups with multiple characteristics had times 40 to 60 minutes longer than patients without these characteristics. For example, an elderly (aged ≥70 years), black, diabetic man or woman arrived 166 or 170 minutes, respectively, after the onset of symptoms compared with 106 minutes for a younger white man without DM. The combination of older age (≥70 years), black or Hispanic race/ethnicity, female sex, and DM represented particularly vulnerable subgroups who exhibited delays of much larger magnitude compared with patients with a single risk factor for delay. Improving patient responsiveness and access in these subgroups represents an important opportunity to decrease adverse consequences from STEMI, improve quality of care, and minimize disparities in care.

To our knowledge, this study is the largest contemporary report from nationally representative data and advances the existing research on the correlates of delay from symptom onset to hospital presentation in several respects. Previous studies1216 have shown that older patients, women, patients identified as black, diabetic patients, and those with atypical symptoms exhibited longer delays. The present study confirmed that delay in hospital presentation was longer for these groups, but the magnitude of these differences (+10 to +30 minutes) after multivariate adjustment was small compared with the duration of delay across the entire cohort, in which nearly half of the patients with STEMI arrived at the hospital more than 2 hours after symptom onset. Although previous studies have reported the odds ratio of individual factors associated with delay, we used a novel approach of showing the cumulative effect of having multiple characteristics in natural units of incremental minutes of delay. We demonstrated that certain patient subgroups with a combination of factors (age, race/ethnicity, sex, and DM) were particularly vulnerable to a delay of up to 60 minutes longer than the reference group. This large cohort of patients with STEMI also allowed us to show the novel finding that patients identified as Hispanic have a delay from symptom onset to hospital arrival comparable to that observed in black patients and have similar effects on times when combined with age, sex, or DM. For example, older men or women (aged ≥70 years) who have DM and are identified as Hispanic had delay times 51 minutes longer than younger men identified as white and without DM.

The geometric mean for delay in hospital presentation decreased from 123 minutes in 1995 to 113 minutes in 2004 and also decreased in some high-risk groups (elderly patients, women, nonwhite patients, and those with noncommercial insurance), which may be related to educational initiatives such as the National Heart, Lung, and Blood Institute’s National Heart Attack Alert Program (http://www.nhlbi.nih.gov/about/nhaap/index.htm). The present study documented that delays from symptom onset to hospital presentation remain common for patients with STEMI, and less than one-third (30.9%) of patients with STEMI arrive at the hospital within 1 hour after the onset of symptoms. These delays highlight the need for improvement strategies because longer delays contribute to longer ischemic times, more myocardial damage, and adverse clinical consequences and decrease the effectiveness of reperfusion therapy, which has the greatest benefit for patients who present with the shortest delay. Recently, there has been great interest in strategies to reduce door-to-balloon time25 and to develop systems of care to transfer and increase the number of patients with STEMI who are eligible to receive primary PCI across large geographic regions.26 These innovative approaches have focused on coordinating and streamlining processes and improving the reliability of systems of care within a hospital and between hospital networks to reduce door-to-balloon time. To optimally use these systems of care, all patients with STEMI must be able to access these systems as soon as possible after the onset of symptoms. Of particular concern from this study was that 8.7% of patients with STEMI presented more than 12 hours after the onset of symptoms, which is beyond the window of eligibility for reperfusion therapy as recommended by current guidelines.27 A previous large randomized trial17 that attempted to decrease times to hospital presentation using an intervention of mass media campaigns for entire, diverse communities was largely unsuccessful. The present study identified specific subgroups who are at greatest risk for delays, and the design and implementation of future interventions must consider how to reach these vulnerable subgroups effectively. Design of effective interventions will also require a deeper understanding of the social, cognitive, and emotional factors that contribute to delay in vulnerable subgroups.28,29 Furthermore, the present study showed that patients who have had previous PCI, MI, CHF, or CABG surgery have shown modest or no decrease in delay in hospital presentation across time, and efforts to improve times should also target patients who remain at risk for future cardiac events and presumably receive regular ongoing care from a physician.

Patients who are older, women, and minorities and those who are uninsured or underinsured have been shown to have disparities in health care access and treatment.30 The present study showed that older patients; women; patients of minority race/ethnicity, including black or Hispanic; and those with noncommercial health insurance had higher proportions of patients with STEMI who presented more than 12 hours after the onset of symptoms. Delays in hospital presentation are concentrated in these vulnerable groups and contribute to disparities in access and treatment for STEMI because fewer patients present within the window of time as recommended by the guidelines to receive and benefit from reperfusion therapy.

The NRMI database has the inherent limitations of any voluntary observational registry, and participating hospitals are more likely to be urban, larger, and equipped with catheterization and cardiac surgical resources.18,19 Time of symptom onset relied on patient recall and also required documentation by each hospital participating in this registry. This study calculated delay using the documented time from symptom onset to time of hospital arrival, and the time of symptom onset was not available in 173 051 patients. When we applied all the other criteria for exclusion that we had applied to the overall study population to this group of 173 051 patients, this cohort became 6026 patients who did not have a time of symptom onset and who did not have any other exclusion criterion. We analyzed the characteristics of these 6026 patients and noted that 41.8% did not have chest pain (Table 4). Also, this cohort had a higher prevalence of characteristics associated with longer delays, including being 70 years and older, nonwhite, female, and diabetic and having noncommercial insurance, compared with the study population.

Table 4.

Characteristics of the Cohort of 6026 Patients Excluded Owing to Lack of Documented Time of Symptom Onseta

Description Study Population, No. (%) (N=482 327) Excluded Cohort, No. (%) (n=6026)
Aged <60 y 196 862 (40.8) 1819 (30.2)
Female (yes) 160 185 (33.2) 2507 (41.6)
Race (black) 24 646 (5.1) 416 (6.9)
Commercial health insurance (yes) 188 991 (39.2) 1848 (30.7)
Diabetes mellitus (yes) 103 485 (21.5) 1727 (28.7)
Chest pain at presentation (yes) 434 997 (90.2) 3509 (58.2)
a

P<.001 for all.

To minimize confounding due to patients who are transferred in from another hospital and to determine the stability of these findings, we performed the analysis for factors associated with delay by including and then excluding patients who were transferred in from another hospital. The mode of transport to the hospital (emergency medical services vs self-transport) was not included in the analysis because the NRMI started to collect these data after 2000.

In conclusion, we found that patient subgroups with a combination of factors, including older age, being female, having Hispanic or black race/ethnicity, and having DM, have particular long times from symptom on-set to hospital presentation that may be up to 60 minutes longer than subgroups without those characteristics. Delays are concentrated in vulnerable groups of patients who have been shown to have disparities in health care access and treatment. Delays in hospital presentation may impact degree of myocardial damage, outcomes, and efficacy of reperfusion therapy; therefore, improving times merits attention and represents an opportunity to improve quality of care for patients with STEMI.

Acknowledgments

Funding/Support: This study was supported by grant R01 HL072575 from the National Heart, Lung, and Blood Institute.

Footnotes

Financial Disclosure: None reported.

Author Contributions: Study concept and design: Ting, Bradley, Roger, and Krumholz. Acquisition of data: Ting. Analysis and interpretation of data: Ting, Bradley, Wang, Lichtman, Nallamothu, Sullivan, Gersh, Curtis, and Krumholz. Drafting of the manuscript: Ting and Gersh. Critical revision of the manuscript for important intellectual content: Ting, Bradley, Wang, Lichtman, Nallamothu, Sullivan, Roger, Curtis, and Krumholz. Statistical analysis: Ting, Bradley, Wang, Lichtman, and Gersh. Administrative, technical, and material support: Ting and Roger. Study supervision: Ting.

Role of the Sponsor: The National Heart, Lung, and Blood Institute had no involvement in the design or conduct of the study, data management or analysis, manuscript preparation, review, or authorization for submission. Genentech approved the study and provided access to the NRMI database at no charge. Genentech did not provide any direct support for the study and was not involved in the study design and conduct, analysis, and interpretation of the data or preparation of the manuscript.

References

  • 1.Newby LK, Rutsch WR, Califf RM, et al. Time from symptom onset to treatment and outcomes after thrombolytic therapy: GUSTO-1 Investigators. J Am Coll Cardiol. 1996;27(7):1646–1655. doi: 10.1016/0735-1097(96)00053-8. [DOI] [PubMed] [Google Scholar]
  • 2.Goldberg RJ, Mooradd M, Guriwitz JH, et al. Impact of time to treatment with tissue plasminogen activator on morbidity and mortality following acute myocardial infarction (NRMI 2) Am J Cardiol. 1998;82(3):259–264. doi: 10.1016/s0002-9149(98)00342-7. [DOI] [PubMed] [Google Scholar]
  • 3.Cannon CP, Gibson CM, Lambrew CT, et al. Relationship of symptom-onset-to-balloon time and door-to-balloon time with mortality in patients undergoing angioplasty for acute myocardial infarction. JAMA. 2000;283(22):2941–2947. doi: 10.1001/jama.283.22.2941. [DOI] [PubMed] [Google Scholar]
  • 4.De Luca G, Suryapranata H, Zijlstra F, et al. Symptom-onset-to-balloon time and mortality in patients with acute myocardial infarction treated by primary angioplasty. J Am Coll Cardiol. 2003;42(6):991–997. doi: 10.1016/s0735-1097(03)00919-7. [DOI] [PubMed] [Google Scholar]
  • 5.De Luca G, Suryapranata H, Ottervanger JP, Antman EM. Time delay to treatment and mortality in primary angioplasty for acute myocardial infarction: every minute of delay counts. Circulation. 2004;109(10):1223–1225. doi: 10.1161/01.CIR.0000121424.76486.20. [DOI] [PubMed] [Google Scholar]
  • 6.Brodie BR, Stone GW, Cox DA, et al. Impact of treatment delays on outcomes of primary percutaneous coronary intervention for acute myocardial infarction: analysis from the CADILLAC trial. Am Heart J. 2006;151(6):1231–1238. doi: 10.1016/j.ahj.2005.07.016. [DOI] [PubMed] [Google Scholar]
  • 7.Brodie BR, Webb J, Cox DA, et al. Impact of time to treatment on myocardial reperfusion and infarct size with primary percutaneous coronary intervention for acute myocardial infarction (from the EMERALD Trial) Am J Cardiol. 2007;99(12):1680–1686. doi: 10.1016/j.amjcard.2007.01.047. [DOI] [PubMed] [Google Scholar]
  • 8.McNamara RL, Wang Y, Herrin J, et al. Effect of door-to-balloon time on mortality in patients with ST-segment elevation myocardial infarction. J Am Coll Cardiol. 2006;47(11):2180–2186. doi: 10.1016/j.jacc.2005.12.072. [DOI] [PubMed] [Google Scholar]
  • 9.Zijlstra F, Patel A, Jones M, et al. Clinical characteristics and outcome of patients with early (<2 h), intermediate (2–4 h) and late (<4 h) presentation treated by primary coronary angioplasty or thrombolytic therapy for acute myocardial infarction. Eur Heart J. 2002;23(7):550–557. doi: 10.1053/euhj.2001.2901. [DOI] [PubMed] [Google Scholar]
  • 10.Gibson CM, Murphy SA, Kirtane AJ, et al. Association of duration of symptoms at presentation with angiographic and clinical outcomes after fibrinolytic therapy in patients with ST-segment elevation myocardial infarction. J Am Coll Cardiol. 2004;44(5):980–987. doi: 10.1016/j.jacc.2004.05.059. [DOI] [PubMed] [Google Scholar]
  • 11.Gersh BJ, Stone GW, White HD, Holmes DR., Jr Pharmacological facilitation of primary percutaneous coronary intervention for acute myocardial infarction: is the slope of the curve the shape of the future? JAMA. 2005;293(8):979–998. doi: 10.1001/jama.293.8.979. [DOI] [PubMed] [Google Scholar]
  • 12.Goldberg RJ, Gurwitz JH, Gore JM. Duration of and temporal trends (1994–1997) in prehospital delay in patients with acute myocardial infarction: the Second National Registry of Myocardial Infarction. Arch Intern Med. 1999;159 (18):2141–2147. doi: 10.1001/archinte.159.18.2141. [DOI] [PubMed] [Google Scholar]
  • 13.Goldberg RJ, Yarzebski J, Lessard D, Gore JM. Decade-long trends and factors associated with time to hospital presentation in patients with acute myocardial infarction: the Worchester Heart Attack Study. Arch Intern Med. 2000;160(21):3217–3223. doi: 10.1001/archinte.160.21.3217. [DOI] [PubMed] [Google Scholar]
  • 14.McGinn AP, Rosamond WD, Goff DC, et al. Trends in prehospital delay time and use of emergency medical services for acute myocardial infarction: experience in 4 US communities from 1987–2000. Am Heart J. 2005;150(3):392–400. doi: 10.1016/j.ahj.2005.03.064. [DOI] [PubMed] [Google Scholar]
  • 15.Luepker RV. Delay in acute myocardial infarction: why don’t they come to the hospital more quickly and what we can do to reduce delay. Am Heart J. 2005;150(3):368–370. doi: 10.1016/j.ahj.2005.05.012. [DOI] [PubMed] [Google Scholar]
  • 16.Goldberg RJ, Steg PG, Sadiq I, et al. Extent of and factors associated with delay to hospital presentation in patients with acute coronary disease: the GRACE Registry. Am J Cardiol. 2002;89(7):791–796. doi: 10.1016/s0002-9149(02)02186-0. [DOI] [PubMed] [Google Scholar]
  • 17.Luepker RV, Raczynski JM, Osganian S, et al. Effect of a community intervention on patient delay and emergency medical service use in acute coronary heart disease. JAMA. 2000;284(1):60–67. doi: 10.1001/jama.284.1.60. [DOI] [PubMed] [Google Scholar]
  • 18.Rogers WJ, Canto JG, Lambrew CT, et al. Temporal trends in the treatment of over 1.5 million patients with myocardial infarction in the US from 1990 through 1999: the National Registry of Myocardial Infarction 1, 2, and 3. J Am Coll Cardiol. 2000;36(7):2056–2063. doi: 10.1016/s0735-1097(00)00996-7. [DOI] [PubMed] [Google Scholar]
  • 19.Every NR, Frederick PD, Robinson M, et al. A comparison of the National Registry of Myocardial Infarction 2 with the Cooperative Cardiovascular Project. J Am Coll Cardiol. 1999;33(7):1886–1894. doi: 10.1016/s0735-1097(99)00113-8. [DOI] [PubMed] [Google Scholar]
  • 20.Bland JM, Altman DG. The use of transformation when comparing two means. BMJ. 1996;312(7039):1153. doi: 10.1136/bmj.312.7039.1153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Bland JM, Altman DG. Transformations, means, and confidence intervals. BMJ. 1996;312(7038):1079. doi: 10.1136/bmj.312.7038.1079. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.King G, Tomz M, Wittenberg J. Making the most of statistical analyses: improving interpretation and presentation. Am J Pol Sci. 1998;44:341–355. [Google Scholar]
  • 23.Nallamothu BK, Bates ER, Herrin J, et al. Times to treatment in transfer patients undergoing primary percutaneous coronary intervention in the United States: National Registry of Myocardial Infarction 3/4 analysis. Circulation. 2005;111 (6):761–767. doi: 10.1161/01.CIR.0000155258.44268.F8. [DOI] [PubMed] [Google Scholar]
  • 24.Magid DJ, Wang Y, Herrin J, et al. Relationship between time of day, day of week, timeliness of reperfusion, and in-hospital mortality for patients with acute ST-segment elevation myocardial infarction. JAMA. 2005;294(7):803–812. doi: 10.1001/jama.294.7.803. [DOI] [PubMed] [Google Scholar]
  • 25.Bradley EH, Herrin J, Wang Y, et al. Strategies for reducing door-to-balloon time in acute myocardial infarction. N Engl J Med. 2006;355(22):2308–2320. doi: 10.1056/NEJMsa063117. [DOI] [PubMed] [Google Scholar]
  • 26.Jacobs AK, Antman EA, Ellrodt G, et al. Recommendation to develop strategies to increase the number of ST-elevation myocardial infarction patients with timely access to primary percutaneous coronary intervention. Circulation. 2006;113 (17):2152–2163. doi: 10.1161/CIRCULATIONAHA.106.174477. [DOI] [PubMed] [Google Scholar]
  • 27.Antman EM, Anbe DT, Armstrong PW, et al. ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Revise the 1999 Guidelines for the Management of Patients With Acute Myocardial Infarction) J Am Coll Cardiol. 2004;44(3):E1–E211. doi: 10.1016/j.jacc.2004.07.014. [DOI] [PubMed] [Google Scholar]
  • 28.Kainth A, Hewitt A, Sowden A, et al. Systematic review of interventions to reduce delay in patients with suspected heart attack. Emerg Med J. 2004;21(4):506–508. doi: 10.1136/emj.2003.013276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Moser DK, Kimble LP, Alberts MJ, et al. Reducing delay in seeking treatment by patients with acute coronary syndrome and stroke. Circulation. 2006;114(2):168–182. doi: 10.1161/CIRCULATIONAHA.106.176040. [DOI] [PubMed] [Google Scholar]
  • 30.Rahimi AR, Spertus JA, Reid KJ, Bernheim SM, Krumholz HM. Financial barriers to health care and outcomes after acute myocardial infarction. JAMA. 2007;297(10):1063–1072. doi: 10.1001/jama.297.10.1063. [DOI] [PubMed] [Google Scholar]

RESOURCES