Abstract
Background
Timely thrombolytic therapy can improve stroke outcomes. Nevertheless, the ability of US hospitals to meet guidelines for intravenous tissue plasminogen activator (tPA) remains suboptimal. What is unclear is whether hospitals accurately perceive their rate of tPA “door-to-needle” (DTN) time within 60 minutes and how DTN rates compare across different hospitals.
Methods and Results
DTN performance was defined by the percentage of treated patients who received tPA within 60 minutes of arrival. Telephone surveys were obtained from staff at 141 Get With The Guidelines hospitals, representing top, middle, and lowDTN performance. Less than one-third (29.1%) of staff accurately identified their DTN performance. Among middle- and low-performing hospitals (n=92), 56 sites (60.9%) overestimated their performance; 42% of middle performers and 85% of low performers overestimated their performance. Sites that overestimated tended to have lower annual volumes of tPA administration (median 8.4 patients [25th to 75th percentile 5.9 to 11.8] versus 10.2 patients [25th to 75th percentile 8.2 to 17.3], P=0.047), smaller percentages of eligible patients receiving tPA (84.7% versus 89.8%, P=0.008), and smaller percentages of DTN ≤60 minutes among treated patients (10.6% versus 16.6%, P=0.002).
Conclusions
Hospitals often overestimate their ability to deliver timely tPA to treated patients. Our findings indicate the need to routinely provide comparative provider performance rates as a key step to improving the quality of acute stroke care.
Keywords: hospital perception, quality improvement, quality of care, stroke, thrombolytics
Stroke is the fourth leading cause of mortality and a leading cause of disability in the United States.1 Early detection and treatment of acute stroke symptoms with intravenous tissue plasminogen activator (tPA) has been demonstrated to improve both short- and long-term outcomes including functional status and quality of life.2 Despite widely disseminated guidelines on patient eligibility criteria and process-of-care metrics, many patients do not receive these recommended interventions in a timely fashion.3 In an effort to optimize stroke care, evidence-based management guidelines and protocols for the establishment of primary stroke centers have been benchmarked nationally through voluntary national registries, such as the American Heart Association’s (AHA’s) Get With The Guidelines (GWTG)–Stroke.4–8 In addition, best practices in stroke care are publicly disseminated to help lagging hospitals improve their care to meet guideline recommendations. Despite these efforts, significant variation and gaps are still present in hospitals’ use and application of AHA process guidelines.3
An element identified as critical to successfully transforming health care delivery is having the impetus to change.9 Overestimation of the quality of care that an institution provides may perpetuate suboptimal performance, whereas accurate measurements of current performance and realistic comparison to other more successful sites might provide the needed motivation to fuel quality improvement.9–12 Prior studies have found that without access to accurate, reliable, and timely comparative data, providers have a tendency to believe they are at least as good as others.9–12 To date, no studies have looked at how hospitals perceive their quality of stroke care relative to other hospitals.
The purpose of this study was to compare stroke teams’ perceptions of their performance against known metrics associated with tPA administration. We surveyed GWTG-Stroke registry institutions to examine the extent to which hospital staff accurately reported both the percentage of eligible patients who received tPA at their site and the percentage of treated patients who received tPA within 60 minutes of arrival. We also assessed the stroke teams’ perceptions of how their performance on guideline-based tPA administration compared with their peers. Finally, we sought to identify hospital factors associated with overestimating “door-to-needle” (DTN) performance.
Methods
The GWTG-Stroke registry is an initiative of the AHA and the American Stroke Association, with the goal of improving the care and outcomes of patients with stroke and transient ischemic attacks. Details of the design and conduct of the GWTG-Stroke program have been published previously.6 GWTG-Stroke uses a Web-based patient management tool (Outcome Sciences, Inc) to collect clinical data on consecutively admitted patients, to provide decision support, and to enable real-time online reporting features.6,13 Hospital personnel collect data on patients admitted with the principal clinical diagnosis of acute stroke or transient ischemic attack by prospective clinical identification, retrospective identification through the use of discharge codes, or a combination thereof. Outcome Sciences, Inc, serves as the data collection and coordination center for GWTG-Stroke. The Duke Clinical Research Institute serves as the GWTG data analysis center and has an agreement to analyze the aggregate deidentified data for research purposes. The Duke University Health System institutional review board determined that this project met the criteria for a declaration of exemption under the Code of Federal Regulations 45 CFR 46.101(b)(2).
Patient data, such as demographics, medical history, onset time of stroke symptoms (recorded as last known well time), arrival time, treatments and procedures, tPA treatment initiation time, and tPA complications, were abstracted from the GWTG-Stroke registry. Data on hospital-level characteristics (eg, bed size, geographical region, teaching status, and annual stroke volume) were gathered from the AHA hospital database.6
Survey Tool Creation and Implementation
A structured survey tool consisting of 70 items regarding components of stroke care management, including communication and teamwork, current process of care, organizational culture, performance monitoring and feedback, and overcoming of barriers was used for this study. The survey was a standardized telephone questionnaire; each item had 5 Likert-type answer options, and respondents chose the option that most accurately fit the site’s status. The survey questions were developed based on a previous qualitative study conducted by our research group and content experts.14 The questions were then reviewed by 10 stroke neurologists, stroke advanced practice nurses, and coordinators—all considered experts in the field. Based on this feedback, the survey was refined and then given to 5 stroke nurse coordinators for additional input. The survey was field tested through telephone interviews with medical and nursing staff at community hospitals, further refined through factor analysis, and then finalized.15 Three questions were created to elicit each site’s perception of tPA administration and DTN performance, which was defined as the percentage of patients who received tPA within 60 minutes of arrival among all patients who received tPA at the institution. Respondents were asked to quantify both their tPA administration rate among eligible patients and their DTN performance as 0% to 20%, 20% to 40%, 40% to 60%, 60% to 80%, or 80% to 100%. They were further asked to rank themselves as a low, below-average, average, above-average, or top performer in terms of their DTN performance on a national level. Accuracy of perception was defined as the respondent’s ability to correctly identify their performance status relative to all GWTG-Stroke hospitals (below average, average, above average). Responses overestimating or underestimating performance were scored as inaccurate (eg, perceiving hospital performance as average when it is actually below average). A factor analysis of the other survey items has been published separately.15
Survey targets were key personnel in stroke care at eligible GWTG-Stroke sites. Contact was established through information from the GWTG-Stroke database (as available) or from AHA quality improvement directors or by direct calling of the hospital. We interviewed a representative from each institution who identified himself or herself as a person familiar with the hospital’s tPA administration process. These interviewees included stroke neurologists, stroke coordinators, and hospital-based quality-improvement staff who were considered experts in the field. Interviews were ≈15 minutes long and were conducted by telephone between January 2011 and July 2011.
Statistical Analysis
Patient sociodemographic and clinical variables, hospital characteristics, and quality-of-care measures were compared among hospitals classified according to performance ranking. Performance rank, categorized as top, middle, and low performers, was considered to be ordinal. Categorical variables were reported as percentages, and continuous variables were reported as medians and 25th and 75th percentiles. Categorical variables were compared using χ2 rank-based group mean-score statistics. Continuous variables were compared using Cochran–Mantel–Haenszel statistics.
Further analysis of sites focused on the dichotomized ranking of their relative performance into overestimation versus no overestimation. Top-performing hospitals were excluded for this analysis because their ranking could not be overestimated. Middle-performing hospitals that answered “above average” or “top performer” were categorized as overestimation, whereas those that answered “average,” “below average,” or “low performer” were categorized as no overestimation. Low-performing hospitals that answered “average,” “above average,” or “top performer” were labeled as overestimation, whereas those that answered “below average” or “low performer” were labeled as no overestimation. Sociodemographic factors, clinical variables, hospital characteristics, and quality-of-care measures were compared between these 2 groups. Pearson χ2 tests and Wilcoxon rank-sum tests were used to compare categorical and continuous variables, respectively. A multivariable logistic regression model was used to determine which factors were independently associated with DTN performance overestimation. Variables that were likely potential confounders were identified a priori to be included in this model based on literature review and clinical judgment. These variables were Joint Commission Primary Stroke Center status, geographic region, annual volume of tPA, annual volume of ischemic stroke, percentage of DTN time within 60 minutes, and years that the survey respondent had been with the site’s stroke team.
All tests were 2-sided with P<0.05 considered as the level of statistical significance. All statistical analyses were performed using SAS version 9.3 (SAS Institute).
Results
All 1395 hospitals participating in GWTG-Stroke between October 1, 2009, and September 30, 2010, were included as potential survey sites. Low-volume sites, defined as those treating <6 acute ischemic stroke patients with tPA during the study period, were excluded from the sampling frame. The remaining hospitals (n=704) were ranked by DTN performance. A total of 300 hospitals were identified for the telephone survey. The 100 highest and 100 lowest performing hospitals from this ranked list were selected, with DTN performance ranging from 45.2% to 92.3% and from 0.0% to 0.0%, respectively. Fifty additional hospitals above and below the median DTN performance of the ranked list (20.0%) were selected to form 100 middle-performing hospitals (DTN performance range 16.7% to 25.0%). Of these 300 sites, 157 responded to initial site contact. Among these, 16 sites were excluded: 2 sites refused to complete a telephone interview, and an additional 14 sites did not respond to key questions or had incomplete data (>10% of responses missing). The final sample represented 141 hospitals that treated 48 201 stroke patients during the study period.
Patient- and hospital-level characteristics were compared between surveyed and nonsurveyed hospitals. Surveyed hospitals tended to have a higher number of eligible patients receiving tPA (P=0.001) and a higher annual tPA volume (P=0.006) but did not have significant differences in other hospital characteristics, including annual stroke volume or rate of DTN ≤60 minutes (Table1). Most patient demographics queried showed statistically significant differences, including race/ethnicity, insurance status, and components of medical history, although many of these differences were modest (Table2).
Table 1.
Hospital Characteristics of Surveyed Versus Nonsurveyed Sites
| Surveyed Hospitals (n=141) | Nonsurveyed Hospitals (n=159) | P Value | |
|---|---|---|---|
| Hospital region, % | |||
| West | 24.1 | 14.5 | 0.023 |
| South | 27.7 | 43.4 | |
| Midwest | 17.7 | 17.0 | |
| Northeast | 30.5 | 25.2 | |
| Rural, % | 1.4 | 5.0 | 0.079 |
| Academic, % | 49.7 | 48.4 | 0.732 |
| TJC Primary Stroke Center, % | 45.4 | 46.5 | 0.842 |
| Annual tPA volume, median (25th to 75th P) | 10.7 (7.5 to 16.0) | 8.2 (5.6 to 15.0) | 0.006 |
| Annual ischemic stroke volume, median (25th to 75th P) | 170.8 (115.0 to 225.6) | 160.7 (102.0 to 230.1) | 0.260 |
| Number of eligible patients treated with tPA, median (25th to 75th P)* | 12.0 (8.0 to 20.0) | 9.0 (6.0 to 15.0) | 0.001 |
| % DTN ≤60 minutes in treated patients, median (25th to 75th P) | 20.0 (0.0 to 50.0) | 16.7 (0.0 to 44.0) | 0.213 |
25th to 75th P indicates 25th to 75th percentile; DTN, door-to-needle; TJC, The Joint Commission; tPA, tissue plasminogen activator.
Data from October 1, 2009, to September 30, 2010.
Table 2.
Patient Characteristics of Surveyed Versus Nonsurveyed Sites
| Surveyed Hospitals (n=141) | Nonsurveyed Hospitals (n=159) | P Value | |
|---|---|---|---|
| Age, median (25th to 75th P), y | 72 (59 to 82) | 71 (59 to 82) | 0.357 |
| Male, % | 47.6 | 46.8 | 0.010 |
| Race/ethnicity, % | |||
| White, non-Hispanic | 70.0 | 70.5 | <0.001 |
| African American | 15.5 | 16.1 | |
| Hispanic | 6.3 | 6.2 | |
| Asian | 3.4 | 2.1 | |
| Other | 1.8 | 0.9 | |
| Insurance status, % | |||
| Medicare | 34.7 | 36.6 | <0.001 |
| Medicaid | 8.6 | 9.5 | |
| Self/none | 6.6 | 6.1 | |
| Private/VA/others | 38.0 | 38.2 | |
| Medical history, % | |||
| Atrial fibrillation/flutter | 17.3 | 17.1 | 0.490 |
| Prosthetic heart valve | 1.4 | 1.5 | 0.026 |
| Previous stroke/TIA | 31.9 | 32.6 | 0.033 |
| CAD/prior MI | 26.5 | 28.3 | <0.001 |
| Carotid stenosis | 4.2 | 5.3 | <0.001 |
| Diabetes mellitus | 31.7 | 32.7 | 0.002 |
| Peripheral vascular disease | 4.3 | 4.8 | <0.001 |
| Hypertension | 80.3 | 81.0 | 0.011 |
| Smoker | 18.5 | 18.7 | 0.478 |
| Dyslipidemia | 43.1 | 43.3 | 0.605 |
| Heart failure | 7.7 | 8.2 | 0.010 |
25th to 75th P indicates 25th to 75th percentile; CAD, coronary artery disease; MI, myocardial infarction; TIA, transient ischemic attack; VA, veterans affairs.
The study population consisted of 49 top performers, 52 middle performers, and 40 low performers. Most personnel interviewed from each hospital were nursing staff (85.0%). On average, respondents had been working with their current stroke teams for 4 years (25th to 75th percentile 2 to 6 years). Comparisons of hospital and patient characteristics among these performance groups are shown in Tables3 and 4. Significant hospital-level factors that varied among the 3 performance groups were volume of tPA administration during the performance evaluation period (October 1, 2009, to September 30, 2010; P<0.001), annual volume of tPA administration (P<0.001), and annual ischemic stroke admissions (P=0.045). Top-performing hospitals tended to have more ischemic stroke admissions per year and administered tPA to a greater number of patients. In addition, most patient-level variables showed statistically significant differences among the 3 performance groups. These variables included median age, gender, race/ethnicity, insurance status, and comorbidities such as previous stroke or transient ischemic attack (Table4).
Table 3.
Hospital Characteristics by Performance on DTN ≤60 Minutes
| Top Performers (n=49) | Middle Performers (n=52) | Low Performers (n=40) | P Value | |
|---|---|---|---|---|
| Hospital region, % | ||||
| West | 32.7 | 25.0 | 12.5 | 0.125 |
| South | 24.5 | 25.0 | 35.0 | |
| Midwest | 12.2 | 19.2 | 22.5 | |
| Northeast | 30.6 | 30.8 | 30.0 | |
| Rural, % | 0.0 | 1.9 | 2.5 | 0.309 |
| TJC Primary Stroke Center, % | 46.9 | 42.3 | 47.5 | 0.998 |
| Annual tPA volume, median (25th to 75th P) | 14.0 (8.5 to 23.3) | 11.3 (8.2 to 17.2) | 7.8 (4.6 to 10.3) | <0.001 |
| Annual ischemic stroke volume, median (25th to 75th P) | 199.0 (107.5 to 286.4) | 174.2 (128.2 to 234.5) | 145.9 (109.5 to 189.9) | 0.045 |
| Number of eligible patients treated with tPA, median (25th to 75th P)* | 15.0 (9.0 to 30.0) | 14.0 (9.0 to 23.0) | 8.0 (5.0 to 12.0) | <0.001 |
| % DTN ≤60 minutes in treated patients, median (25th to 75th P) | 57.1 (50.0 to 65.5) | 18.5 (14.9 to 23.1) | 0.0 (0.0 to 0.0) | <0.001 |
| Title, % | ||||
| Nurse | 77.6 | 94.2 | 82.5 | 0.216 |
| Neurologist | 4.1 | 0.0 | 5.0 | |
| Administrator | 2.0 | 0.0 | 7.5 | |
| Other | 14.3 | 5.8 | 5.0 | |
| Years with current stroke team, median (25th to 75th P) | 4.0 (3.0 to 6.0) | 4.0 (2.5 to 6.0) | 3.0 (2.0 to 5.0) | 0.124 |
25th to 75th P indicates 25th to 75th percentile; DTN, door-to-needle; TJC, The Joint Commission; tPA, tissue plasminogen activator.
Data from October 1, 2009, to September 30, 2010.
Table 4.
Baseline Patient Characteristics by Performance on DTN ≤60 Minutes
| Top Performers (n=49) | Middle Performers (n=52) | Low Performers (n=40) | P Value | |
|---|---|---|---|---|
| Age, median (25th to 75th P), y | 70 (58 to 81) | 73 (60 to 83) | 72 (59 to 82) | <0.001 |
| Male, % | 49.0 | 47.3 | 46.0 | <0.001 |
| Race/ethnicity, % | ||||
| White, non-Hispanic | 64.1 | 75.6 | 70.2 | <0.001 |
| African American | 15.8 | 12.7 | 19.1 | |
| Hispanic | 7.9 | 5.8 | 4.8 | |
| Asian | 4.6 | 2.9 | 2.6 | |
| Other | 3.1 | 1.1 | 1.0 | |
| Insurance status, % | ||||
| Medicare | 38.5 | 31.0 | 34.7 | <0.001 |
| Medicaid | 8.6 | 8.9 | 8.4 | |
| Self/none | 7.5 | 6.4 | 5.6 | |
| Private/VA/others | 31.4 | 46.5 | 34.7 | |
| Medical history, % | ||||
| Atrial fibrillation/flutter | 15.9 | 18.4 | 17.5 | <0.001 |
| Prosthetic heart valve | 1.1 | 1.5 | 1.5 | 0.002 |
| Previous stroke/TIA | 29.3 | 33.0 | 33.9 | <0.001 |
| CAD/prior MI | 26.1 | 26.6 | 26.8 | 0.217 |
| Carotid stenosis | 3.4 | 4.5 | 4.8 | <0.001 |
| Diabetes mellitus | 31.6 | 31.3 | 32.5 | 0.182 |
| Peripheral vascular disease | 3.8 | 4.6 | 4.4 | 0.003 |
| Hypertension | 78.7 | 81.3 | 80.9 | <0.001 |
| Smoker | 20.2 | 17.5 | 17.6 | <0.001 |
| Dyslipidemia | 39.5 | 47.4 | 41.8 | <0.001 |
| Heart failure | 7.1 | 8.3 | 7.7 | 0.013 |
25th to 75th P indicates 25th to 75th percentile; CAD, coronary artery disease; DTN, door-to-needle; MI, myocardial infarction; TIA, transient ischemic attack; VA, veterans affairs.
Almost two-thirds of respondents (65.3%) correctly estimated the percentage of eligible patients at their institution who received tPA. Figure1 shows a significant trend toward lower accuracy on this measure as DTN performance declined (P=0.002). Regardless of performance category, respondents reported less accurately on their performance on the DTN ≤60 minutes metric (Figure1). Overall, 29.1% of hospitals were able to accurately quantify their DTN ≤60 minutes ability, and this rate of accuracy was not statistically different among performance groups (P=0.853).
Figure 1.
Perception of performance on tPA administration and rates of DTN ≤60 minutes. Percentage of accurate responses regarding tPA administration and rates of DTN ≤60 minutes at respondent’s home institution. DTN indicates door-to-needle; tPA, tissue plasminogen activator.
Although the median DTN ≤60 minutes rate among top-performing hospitals was 57.1% of treated patients, the majority (67.4%) of top-performing hospitals estimated their performance on DTN ≤60 minutes to be >60% (Figure2). Similarly, 67.5% of low-performing hospitals perceived their performance to be >20% (actual median DTN ≤60 minutes rate 0.0%). The majority (81.6%) of top performers recognized themselves as above average or top in their ability to deliver timely tPA relative to other sites (Figure3). In contrast, low performers tended to overestimate their relative ability to provide timely tPA compared with others: 85.0% reported their DTN ≤60 minutes performance to be average, above average, or top in comparison to other US hospitals.
Figure 2.
Perception of performance on door-to-needle (DTN) ≤60 minutes by quartile for hospital ranking. Distribution of responses regarding perceived achievement rate of DTN ≤60 minutes at respondent’s home institution divided by hospital ranking, based on actual performance data, and comparison of response distribution with actual median within performance subgroups. *Indicates category for median rate of actual DTN ≤60 minutes for each classification of performers.
Figure 3.
Perceived relative performance on door-to-needle (DTN) ≤60 minutes by hospital ranking. Distribution of responses regarding perceived success rate of DTN ≤60 minutes at respondent’s home institution relative to other US institutions divided by hospital ranking based on actual performance data.
Among all middle- and low-performing hospitals surveyed (n=92), respondent overestimation occurred at 56 sites (60.9%). Patient and hospital characteristics differed slightly for sites that overestimated their relative performance versus those that did not. Hospitals with staff that overestimated performance had higher percentages of minority patients; slightly younger median age populations; and higher combined percentages of patients on Medicaid, Medicare, or with no insurance. Hospitals with staff that reported accurately or that underestimated their performance relative to others had a slightly higher percentage of patients with almost every medical condition collected. Statistically significant percentages between the 2 groups were found for patients with a history of atrial fibrillation or flutter, coronary artery disease or previous myocardial infarction, peripheral vascular disease, smoking, dyslipidemia, and heart failure. At hospitals with staff that were accurate or that underestimated their performance, higher percentages of patients were on antiplatelet medications, cholesterol-lowering medications, and anticoagulation medications prior to admission.
Sites with overestimated DTN performance tended to have lower volumes of tPA administration during the performance evaluation period (median 10.0 [25th to 75th percentile 8.0 to 15.5] versus median 14.5 [25th to 75th percentile 9.0 to 22.5], P=0.022), smaller percentages of eligible patients receiving tPA (84.7% versus 89.8%, P=0.008), and smaller percentages of DTN ≤60 minutes (10.6% versus 16.6%, P=0.002) (Table5). Other hospital characteristics, such as hospital size, Joint Commission Primary Stroke Center status, academic affiliation, geographic region, and hospital location (rural versus urban) were not found to be associated with accuracy of performance perception.
Table 5.
Hospital Characteristics of Overestimation Versus No Overestimation
| Overestimation (n=56) | No Overestimation (n=36) | P Value | |
|---|---|---|---|
| Hospital region, % | |||
| West | 17.9 | 22.2 | 0.415 |
| South | 35.7 | 19.4 | |
| Midwest | 19.6 | 22.2 | |
| Northeast | 26.8 | 36.1 | |
| Rural, % | 1.8 | 2.8 | 0.736 |
| TJC Primary Stroke Center, % | 46.4 | 41.7 | 0.656 |
| Annual tPA volume, median (25th to 75th P) | 8.4 (5.9 to 11.8) | 10.2 (8.2 to 17.3) | 0.047 |
| Annual ischemic stroke volume, median (25th to 75th P) | 163.1 (111.9 to 219.2) | 166.5 (128.4 to 209.6) | 0.609 |
| Administered tPA among eligible patients | 84.7 | 89.8 | 0.008 |
| DTN ≤60 minutes in treated patients, % | 10.6 | 16.6 | 0.002 |
| Title, % | |||
| Nurse | 85.7 | 94.4 | 0.339 |
| Neurologist | 3.6 | 0.0 | |
| Administrator | 5.4 | 0.0 | |
| Others | 5.4 | 5.6 | |
| Years with current stroke team, median (25th to 75th P) | 4.0 (2.0 to 5.5) | 4.0 (2.5 to 6.0) | 0.731 |
| Percentage of hospitals accurately reporting eligible patients who receive tPA, % | 58.9 | 58.3 | 0.955 |
| Percentage of hospitals accurately reporting treated patients who receive tPA <60 minutes, % | 23.2 | 33.3 | 0.290 |
25th to 75th P indicates 25th to 75th percentile; DTN, door-to-needle; TJC, The Joint Commission; tPA, tissue plasminogen activator.
In a multivariable analysis of factors associated with relative DTN performance overestimation, sites were less likely to overestimate their performance for every 10-case increase in annual tPA volume (odds ratio 0.31, 95% CI 0.10 to 0.94) and for every 10-percentage-point increase in the percentage of DTN times within 60 minutes of arrival (odds ratio 0.59, 95% CI 0.36 to 0.96). Other factors were not significantly associated with overestimation of relative performance. These results are shown in Table6.
Table 6.
Multivariable Model of Overestimation
| Variable | OR | 95% CI | P Value |
|---|---|---|---|
| TJC Primary Stroke Center | 1.15 | (0.41 to 3.17) | 0.792 |
| Geographic region | |||
| Northeast | 1.00 (ref) | — | — |
| Midwest | 1.03 | (0.28 to 3.71) | 0.970 |
| South | 1.76 | (0.50 to 6.22) | 0.380 |
| West | 1.62 | (0.40 to 6.46) | 0.498 |
| Annual volume of tPA administration, per 10-case increase | 0.31 | (0.10 to 0.94) | 0.039 |
| Annual volume of ischemic stroke admissions, per 50-case increase | 1.38 | (0.91 to 2.10) | 0.128 |
| Percentage of DTN within 60 minutes during the study period, per 10-percentage-point increase | 0.59 | (0.36 to 0.96) | 0.035 |
| Number of years survey respondent had been with the site’s stroke team | 1.01 | (0.89 to 1.15) | 0.849 |
DTN indicates door-to-needle; OR, odds ratio; TJC, The Joint Commission; tPA, tissue plasminogen activator.
Discussion
Our study demonstrates that staff perception of their own hospital’s performance in terms of timeliness of tPA administration in acute stroke care is generally more optimistic. Less than one-third of identified hospital-based stroke-care quality leaders could accurately quantify their institution’s rate of DTN ≤60 minutes. Furthermore, hospital personnel tended to overestimate their DTN performance relative to peers. Among low-performing hospitals, 85% overestimated their institution’s performance of DTN time, with almost 5% of low performers believing their DTN performance was “superior” on a national level. Sites with higher annual volumes of tPA administration and higher percentages of DTN times within 60 minutes were less likely to overestimate their relative DTN performance.
We found that institutions often did not recognize their underperformance relative to other sites, and that is consistent with prior studies. A survey of US hospital board chairs found that two-thirds of respondents rated their institution’s Joint Commission or Hospital Quality Alliance performance as better or much better than that of other hospitals. Among low-performing hospitals, 58% reported their performance to be better or much better than the national average; none of these hospitals reported their performance to be worse or much worse than their US peers.10 A Swedish survey by Pukk et al found similar results; respondents tended to rate their institution as equal to or better than others.12 Nonetheless, previous studies have noted that the impetus to change is 1 of the top 5 elements critical to the successful and sustainable transformation of patient care.16 Consequently, addressing misperceptions that one’s performance is average or above par when it actually is not is an important step in addressing this motivation for change. In addition, previous studies have demonstrated that having leadership and a culture of continuous quality improvement, as well as a financial commitment to quality, are highly associated with hospital improvement in the care of patients with acute coronary syndromes.17 Providing service-line directors, nursing analytic tools, and comparative data further assist in perpetuating interest in continuous quality improvement.18
A study of hospital board chairs showed that chairs of top-performing hospitals were more likely to report being familiar with quality measures and to regularly review performance reports.10 Prior systematic reviews have also demonstrated the impact of audit and feedback on health care professionals’ compliance with desired practice and shown that feedback is more effective when baseline performance is low.19 Consequently, it appears that with access to comparative data, respondents are more realistic about their performance, whereas in instances in which no or little data were available, respondents tended to overestimate their own performance.12,20
In our study, overall stroke volume and tPA volume were related to performance of DTN time, and top performers were more accurate in their perceptions of performance, suggesting that they may track this information and that these metrics are important. In contrast, lower performers overestimated their institution’s performance of tPA use relative to peers. This gap between performance and perception could be the result of better communication of internal and relative performance at high-performing hospitals; this communication may increase awareness and engage all stroke-care staff in quality improvement. Understanding these differences will help improve stroke care for all hospitals.
Our study provides further insights into the challenges of feedback and provider self-reflection. Critical to the success and interest in GWTG-Stroke as a voluntary quality-improvement initiative is the ability for hospitals to submit their institutional stroke care data and to have this information benchmarked against similar peers. The basis for the program is to provide hospital physicians and leadership with the ability to assess their quality-improvement efforts relative to secular and policy trends that influence all hospitals’ care of stroke patients. All institutions in our study were able to download routine site-specific performance data, yet it is unclear how individual hospitals used this information and shared metrics among personnel in stroke care for assessment of quality gaps and case review. The striking discordance among all hospitals and the care provided suggests that opportunities exist to focus communication of individual performance and relative performance across personnel involved in stroke care. Even though our survey efforts selected a single member of a larger team or a person with limited experience, training or exposure to these data and the fragmentation in the understanding of relative performance are still reflected. Further studies are needed to evaluate practices used by stroke teams to stay abreast of their performance on a national scale and will help implementation of targeted interventions to address the misperceptions identified through our research.
Limitations
Several limitations of this study need to be acknowledged. First, only GWTG-Stroke hospitals were included in this survey. Participation in the GWTG-Stroke registry is voluntary; therefore, despite participation from >1000 hospitals across the United States, this study could select centers with greater interest in stroke-care quality improvement. Consequently, the accuracy of perception of quantitative performance may not be applicable to non–GWTG-Stroke hospitals. Extrapolation of these results on perception of relative performance may also be inaccurate. Second, as with all surveys, the data presented are dependent on the subjective interpretation and biases of interview participants. Furthermore, only 1 representative was interviewed from each institution; obtaining perceptions from a larger number of team members from each hospital may have produced different findings. Surveys and interview formats were standardized to minimize variance among interviews. Third, although our response rate was quite high among those we were able to successfully contact (141 of 157; 89%), we failed to find correct contact information for many sites that were originally selected. Finally, although hospitals were identified based on their performance in 2009–2010, interviews were conducted between January 2011 and July 2011, and that may have created some historical bias. Quality-improvement initiatives implemented during this time lag may have affected both measured and perceived performance.
Conclusion
Personnel at low-performing hospitals had a tendency to overestimate their performance in timely tPA administration in stroke care relative to other sites participating in GWTG-Stroke. Such overestimation may have important implications for the low motivation to investigate improvement barriers and to implement site-specific quality-improvement initiatives. Further research is needed to identify the reasons for this misperception in performance. Potential changes in benchmarked performance feedback reports should be considered to ensure broader distribution of improvement initiatives.
Acknowledgments
The authors would like to thank Erin Hanley, MS, for her editorial contributions to this manuscript. Hanley did not receive compensation for her contributions, apart from her employment at the institution where this study was conducted.
Sources of Funding
This project was supported by cooperative agreement number U18HS016964 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
Disclosures
Dr Lin, Dr Olson, Dr Britz, and Mr Constable have no relevant disclosures to report. Ms Cox is an employee of the Duke Clinical Research Institute, which serves as the AHA GWTG data coordinating center. Dr Fonarow receives research support from the National Institutes of Health (significant), and is an employee of the University of California, which holds a patent on retriever devices for stroke (significant). Dr Schwamm serves as chair of the AHA GWTG Stroke Clinical Workgroup; serves as a consultant to the Massachusetts Department of Public Health and serves on the Steering Committee for Lundbeck’s DIAS4 clinical trial. He is PI of an NINDS-funded trial of extended window thrombolysis for which Genentech provides alteplase free of charge and some supplemental site payments. Dr Peterson has received research grants from Lilly, Johnson & Johnson, and Bristol-Myers Squibb, Sanofi-Aventis, and Merck-Schering Plough partnership. Dr Peterson serves as Principal Investigator (PI) of the Data Analytic Center for American Heart Association’s Get With the Guidelines and the PI for the CERTs program, which funded this study. Dr Shah reports being a consultant for Castlight Health, LLC.
References
- Towfighi A, Saver JL. Stroke declines from third to fourth leading cause of death in the United States. Stroke. 2011;42:2351–2355. doi: 10.1161/STROKEAHA.111.621904. [DOI] [PubMed] [Google Scholar]
- Hacke W, Donnan G, Fieschi C, Kaste M, von Kummer R, Broderick JP, Brott T, Frankel M, Grotta JC, Haley EC, Jr, Kwiatkowski T, Levine SR, Lewandowski C, Lu M, Lyden P, Marler JR, Patel S, Tilley BC, Albers G, Bluhmki E, Wilhelm M, Hamilton S ATLANTIS Trials Investigators; ECASS Trials Investigators; NINDS rt-PA Study Group Investigators. Association of outcome with early stroke treatment: pooled analysis of ATLANTIS, ECASS, and NINDS rt-PA stroke trials. Lancet. 2004;363:768–774. doi: 10.1016/S0140-6736(04)15692-4. [DOI] [PubMed] [Google Scholar]
- Fonarow GC, Smith EE, Saver JL, Reeves MJ, Bhatt DL, Grau-Sepulveda MV, Olson DM, Hernandez AF, Peterson ED, Schwamm LH. Timeliness of tissue-type plasminogen activator therapy in acute ischemic stroke: patient characteristics, hospital factors, and outcomes associated with door-to-needle times within 60 minutes. Circulation. 2011;123:750–758. doi: 10.1161/CIRCULATIONAHA.110.974675. [DOI] [PubMed] [Google Scholar]
- Smaha LA American Heart Association. The American Heart Association Get With The Guidelines program. Am Heart J. 2004;148:S46–S48. doi: 10.1016/j.ahj.2004.09.015. [DOI] [PubMed] [Google Scholar]
- Fonarow GC, Reeves MJ, Smith EE, Saver JL, Zhao X, Olson DW, Hernandez AF, Peterson ED, Schwamm LH GWTG-Stroke Steering Committee and Investigators. Characteristics, performance measures, and in-hospital outcomes of the first one million stroke and transient ischemic attack admissions in Get With The Guidelines-Stroke. Circ Cardiovasc Qual Outcomes. 2010;3:291–302. doi: 10.1161/CIRCOUTCOMES.109.921858. [DOI] [PubMed] [Google Scholar]
- Schwamm LH, Fonarow GC, Reeves MJ, Pan W, Frankel MR, Smith EE, Ellrodt G, Cannon CP, Liang L, Peterson E, Labresh KA. Get With The Guidelines-Stroke is associated with sustained improvement in care for patients hospitalized with acute stroke or transient ischemic attack. Circulation. 2009;119:107–115. doi: 10.1161/CIRCULATIONAHA.108.783688. [DOI] [PubMed] [Google Scholar]
- Adams HP, Jr, del Zoppo G, Alberts MJ, Bhatt DL, Brass L, Furlan A, Grubb RL, Higashida RT, Jauch EC, Kidwell C, Lyden PD, Morgenstern LB, Qureshi AI, Rosenwasser RH, Scott PA, Wijdicks EF American Heart Association; American Stroke Association Stroke Council; Clinical Cardiology Council; Cardiovascular Radiology and Intervention Council; Atherosclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups. Guidelines for the early management of adults with ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council, Clinical Cardiology Council, Cardiovascular Radiology and Intervention Council, and the Atherosclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups: the American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists. Stroke. 2007;38:1655–1711. doi: 10.1161/STROKEAHA.107.181486. [DOI] [PubMed] [Google Scholar]
- Alberts MJ, Hademenos G, Latchaw RE, Jagoda A, Marler JR, Mayberg MR, Starke RD, Todd HW, Viste KM, Girgus M, Shephard T, Emr M, Shwayder P, Walker MD. Recommendations for the establishment of primary stroke centers. JAMA. 2000;283:3102–3109. doi: 10.1001/jama.283.23.3102. . Brain Attack Coalition. [DOI] [PubMed] [Google Scholar]
- Shwartz M, Cramer IE, Holmes SK, Cohen AB, Restuccia JD, VanDeusen Lukas C, Sullivan JL, Charns MP. Survey-assessed quality and organizational factors related to quality in Pursuing Perfection hospitals. Qual Manag Health Care. 2010;19:349–363. doi: 10.1097/QMH.0b013e3181f9ef02. [DOI] [PubMed] [Google Scholar]
- Jha A, Epstein A. Hospital governance and the quality of care. Health Aff (Millwood) 2010;29:182–187. doi: 10.1377/hlthaff.2009.0297. [DOI] [PubMed] [Google Scholar]
- Ginsburg LS. Factors that influence line managers’ perceptions of hospital performance data. Health Serv Res. 2003;38:261–286. doi: 10.1111/1475-6773.00115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pukk K, Lundberg J, Gaffney A, Penaloza-Pesantes RV, Olsson J. Do health care managers know the comparative quality of their care? Qual Manag Health Care. 2003;12:232–239. doi: 10.1097/00019514-200310000-00008. [DOI] [PubMed] [Google Scholar]
- LaBresh KA, Reeves MJ, Frankel MR, Albright D, Schwamm LH. Hospital treatment of patients with ischemic stroke or transient ischemic attack using the “Get With The Guidelines” program. Arch Intern Med. 2008;168:411–417. doi: 10.1001/archinternmed.2007.101. [DOI] [PubMed] [Google Scholar]
- Olson DM, Constable M, Britz GW, Lin CB, Zimmer LO, Schwamm LH, Fonarow GC, Peterson ED. A qualitative assessment of practices associated with shorter door-to-needle time for thrombolytic therapy in acute ischemic stroke. J Neurosci Nurs. 2011;43:329–336. doi: 10.1097/JNN.0b013e318234e7fb. [DOI] [PubMed] [Google Scholar]
- Olson DM, Cox M, Constable M, Britz GW, Lin CB, Zimmer LO, Fonarow GC, Schwamm LH, Peterson ED. Development and initial testing of the stroke rapid-treatment readiness tool. J Neurosci Nurs. 2014;46:267–273. doi: 10.1097/JNN.0000000000000082. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lukas CV, Holmes SK, Cohen AB, Restuccia J, Cramer IE, Shwartz M, Charns MP. Transformational change in health care systems: an organizational model. Health Care Manage Rev. 2007;32:309–320. doi: 10.1097/01.HMR.0000296785.29718.5d. [DOI] [PubMed] [Google Scholar]
- Glickman SW, Boulding W, Staelin R, Mulgund J, Roe MT, Lytle BL, Rumsfeld JS, Gibler WB, Ohman EM, Schulman KA, Peterson ED. A framework for quality improvement: an analysis of factors responsible for improvement at hospitals participating in the Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes with Early Implementation of the ACC/AHA Guidelines (CRUSADE) quality improvement initiative. Am Heart J. 2007;154:1206–1220. doi: 10.1016/j.ahj.2007.08.001. [DOI] [PubMed] [Google Scholar]
- Render ML, Freyberg RW, Hasselbeck R, Hofer TP, Sales AE, Deddens J, Levesque O, Almenoff PL. Infrastructure for quality transformation: measurement and reporting in veterans administration intensive care units. BMJ Qual Saf. 2011;20:498–507. doi: 10.1136/bmjqs.2009.037218. [DOI] [PubMed] [Google Scholar]
- Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259. doi: 10.1002/14651858.CD000259.pub3. . doi: 10.1002/14651858.CD000259.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Taylor SE, Brown JD. Positive illusions and well-being revisited: separating fact from fiction. Psychol Bull. 1994;116:21–27. doi: 10.1037/0033-2909.116.1.21. discussion 28. [DOI] [PubMed] [Google Scholar]