Improved in-hospital outcomes and care for patients in stroke research

Abstract

Objective:

To describe stroke research activity in Australian acute public hospitals and determine if participation in research provides better quality of care and outcomes for patients with stroke.

Methods:

This was an observational study using data from hospitals that participated in the National Stroke Foundation (Australia) acute services audit program in 2009, 2011, and 2013. This included self-reported organizational features and a retrospective clinical audit of up to 40 medical records of patients with stroke from each hospital. Multilevel random effects logistic regression with level defined as hospital and adjustments for hospital, demographic, clinical, and stroke severity factors were undertaken.

Results:

A total of 240 hospitals submitted organizational data. Hospitals with a stroke unit (70% vs 7%, p < 0.001) and >200 stroke admissions per year (80% vs 17%, p < 0.001) reported greater involvement in research studies. Of 9,537 patients audited at 129 hospitals, 469 (5%) consented to participate in research. Patients who participated in research compared to nonparticipants were likely to be younger (median age 73 years; 25th percentile [Q1]: 63, 75th percentile [Q3]: 80, vs median age 76 years Q1: 64, Q3: 83; p < 0.001) and receive important clinical practices such as a swallow screen/assessment prior to oral intake (62% vs 56%; p < 0.01). An independent association with reduced in-hospital mortality (adjusted odds ratio 0.30, 95% confidence interval 0.12, 0.76) was evident if participating in research regardless of access to stroke unit care.

Conclusions:

Patients who participate in stroke research receive better in-hospital care and are more likely to survive compared to nonresearch participants.

Classification of evidence:

This study provides Class III evidence that patients with stroke who participate in research receive better quality of care and have reduced in-hospital mortality.

Well-designed research trials are essential to advance evidence-based treatment in health care and improve health outcomes for future patients.¹ Clinical research has the potential to directly influence the patient who consents to participate.² This may be detrimental, due to exposure to additional experimental risks,³ or beneficial, from possible positive effects of the intervention, closer monitoring, and implementation of protocols,⁴ or merely from being provided with greater attention and being studied (i.e., the Hawthorne effect).⁵ Determining if this trial effect for patients exists regardless of randomization may have important implications for motivation, not only of hospital teams to become involved in research, but also for patients who are invited to participate.

Although heterogeneity of studies and variation in inclusion criteria exist, several reviews, within a range of disease groups, have been undertaken to determine if a trial effect exists. Few studies have demonstrated positive outcomes for patients participating in trials.^4,6 The majority of published reviews have been inconclusive in providing evidence of harm or benefit related to participation in trials.^7–10

The equivocal evidence presented has not included studies involving the stroke population. Therefore, we believed it was important to describe differences in hospital-based outcomes among patients with stroke participating in research compared to those not participating in research. The National Stroke Foundation (NSF) Audit Program is undertaken voluntarily by staff at acute hospitals around Australia biennially.¹¹ These data provided the opportunity to review clinical care and hospital outcomes of patients involved in research studies at a national level.

Our aims were to describe stroke research activity in Australian acute care hospitals and assess if patients with acute stroke who participate in research were more likely to receive recommended processes of care and experience better in-hospital outcomes compared with patients who did not participate in research studies.

METHODS

Data were obtained from hospitals participating in the NSF Acute Services Audit Program cycles conducted in 2009, 2011, and 2013.^12–17 Participation in the audit was voluntary. The Audit Program has 2 components. The first is a self-reported survey completed by a nominated clinician at each hospital that captures organizational aspects of the service, including bed numbers, admissions/year, and available resources. In addition, how many and the type of current stroke research studies are recorded. These studies are classified according to therapeutic areas, i.e., acute, rehabilitation, prevention, or other.¹⁸ The second aspect of the program is a clinical medical record audit. Trained data abstractors from each hospital retrospectively audit medical records of up to 40 consecutive patients with a primary diagnosis of stroke (ICD-10 codes: 161, I62.9, I63, I64) admitted from July to December the previous year. Case numbers are influenced by hospital stroke admissions. Larger hospitals are encouraged to provide more cases so their data are more representative for their site. Comprehensive methods for the Audit Program have been published previously.¹¹ Briefly, detailed information including patient demographics, adherence to recommended processes of care, hospital outcomes (approximately 7–10 days after stroke), and whether or not patients had consented to be part of a research study (i.e., clinical audit question: “Has the patient consented to participate in a research study?”) are collected on a specially designed Web-based tool using standardized procedures.

Statistical analysis.

Pooled clinical data from hospitals that audited 10 or more cases in total over the 3 audit cycles were included in the analysis. To minimize information bias, patients documented as receiving palliative care were excluded. Only valid yes/no responses were included in the analyses for data related to medical history and the presence of symptoms on presentation to hospital. For data relating to processes of care, i.e., received care in a stroke unit (SU), not documented and unknown responses were assumed to be negative and included in the denominator. Time-sensitive variables (i.e., brain scan within 24 hours) were derived, with unknown times assumed to be outside the nominated time frame and included in the denominator. In the 2013 audit, response options to the clinical question regarding participation in research studies changed from yes/no to yes/no/not documented. For consistent comparisons, not documented responses from the 2013 audit were considered negative and included in the denominator. Univariable analyses were performed to determine differences between research participants and nonresearch participants using χ² tests for categorical variables and Fisher exact test for dichotomous variables with small expected frequencies. The nonparametric Wilcoxon Mann-Whitney rank sum test was used for continuous variables not normally distributed.

Multilevel random effects logistic regression analyses were undertaken for the following outcomes: in-hospital death, independence on discharge measured using the modified Rankin Scale score (0–2),¹⁹ and discharge destination, including to home, in-patient rehabilitation, or an aged care facility. Level was defined by hospital site. For the continuous dependent variable of length of stay, a median regression model with bootstrap estimated standard errors was reported. A parsimonious approach to model development was used. Independent variables included hospital location, hospital stroke admissions, patient characteristics with statistical significance (p < 0.1), and variables considered to be clinically important (i.e., sex). In addition, other confounders including stroke type (ischemic vs intracerebral hemorrhage and unknown) and severity factors such as inability to walk, arm weakness, and speech impairment on admission and incontinence within 72 hours, which are based on a validated prognostic model for comparing patient outcomes,²⁰ were included. Sensitivity analyses were undertaken to further examine the relationship between participation and nonparticipation in research: (1) including SU care as an independent variable, (2) including patients who received palliative care, (3) excluding unknown/not documented/missing responses for process indicators. Subanalyses were undertaken to further explore the potential influence of having received SU care or thrombolysis and included (1) only those who received SU care and (2) only patients with ischemic stroke, with thrombolysis included as an independent variable in the model. Standard techniques were implemented to check for collinearity and the fit of various models were compared using Bayesian information criteria. Values of p < 0.05 were considered significant for all analyses. Adjusted odds ratio (aOR) and coefficients with 95% confidence intervals (CIs) were calculated. Stata 12.0 (Stata Corp., College Station, TX) statistical software was used for all analyses.

Standard protocol approvals, registrations, and patient consents.

Ethics approval was granted through Monash University Human Research Ethics Committee (CF15/3162-2015001349).

Classification of evidence.

This observational study, using audit data collected across 3 cycles, provides Class III evidence that patients with stroke who participate in research receive better quality of care, including access to SUs (83% vs 57%), thrombolysis treatment (18% vs 5%), and timely physiotherapy (73% vs 64%) and speech therapy (73% vs 63%), and have reduced in-hospital mortality (aOR 0.30, 95% CI 0.12–0.76), vs those who do not participate in research.

RESULTS

Characteristics of the hospitals participating in stroke research.

A total of 240 hospitals contributed 571 organizational survey responses over the 3 audit cycles, with 196 hospitals completing the survey in more than one cycle. The majority of responses were from public hospitals (98%). Over time, a higher proportion of hospitals reported to be participating in stroke research (2009: 59/206 [29%]; 2011: 57/188 [30%]; 2013: 72/177 [41%]; p = 0.03). Stroke research was more likely to be conducted in hospitals with a SU (70% vs 7%; p < 0.001), with more than 200 stroke admissions per year (80% vs 17%; p < 0.001), and those located in urban areas (56% vs 7%; p < 0.001). Research involving acute stroke care (i.e., intervention studies within 48 hours of stroke onset) was commonly reported (54%) (n = 365). A further 122 (18%) studies were focused on stroke prevention and 90 (13%) studies concentrated on stroke rehabilitation (i.e., studies recruiting after 48 hours of stroke onset).

Findings from the clinical audit of patient medical records.

There were 129 hospitals that provided clinical audit data for 10,542 cases over the 3 audit cycles, with 111 hospitals completing more than 1 audit. Exclusions included 1,005 (10%) patients who were palliated during their admission. Overall, 469 (5%) patients consented to participate in research at these hospitals, with similar proportions evident across audit cycles (2009: 125 [4%]; 2011: 180 [6%]; 2013: 164 [5%]).

Research participants vs nonresearch participants.

Patients who participated in research studies compared to nonparticipants were younger (median age 73 years; 25th percentile [Q1]: 63, 75th percentile [Q3]: 80, vs median age 76; Q1: 64, Q3: 83; p < 0.001) and more independent prior to their stroke (68% vs 59%; p < 0.001) (table 1).

Table 1.

Results of univariable analyses summarizing patient characteristics, stroke type, severity, and discharge variables for research and nonresearch participants

Research participants were more likely to receive SU care (83% vs 57%; p < 0.001), have a swallow screen/assessment prior to oral intake (62% vs 56%; p < 0.05), receive thrombolysis treatment (18% vs 5%; p < 0.001), and see a physiotherapist (73% vs 64%; p < 0.001) or speech therapist (73% vs 63%; p < 0.001) within 48 hours of admission compared to nonparticipants. Education about behavior change to reduce future stroke risk also occurred more often in research participants (61% vs 45%; p < 0.001) (figure). Overall, not documented/unknown responses for process variables were minimal (0.09%–1.95%). For time-sensitive variables, the occurrence of unknown times ranged from 5% to 32%. Results of the sensitivity analyses excluding unknown/not documented/missing data for adherence to processes were consistent with the main results presented.

Figure. Processes of care received by research and nonresearch participants.

^aBefore food/drink or oral medication. ^bFor patients with ischemic stroke. ^cDeveloped with input from patient/family and team.

Univariable results indicated that research participants had a longer length of stay (LOS) (table 1). However, over the cycles, there was an average 9-hour reduction in LOS per year of audit for all cases (p < 0.001). After adjusting for confounders, including year of audit, no difference in LOS between groups was evident (coefficient 0.44, 95% CI −0.2 to 1.1).

Multivariable results showed an independent association between being enrolled in research and reduced in-hospital mortality (aOR 0.30, 95% CI 0.12–0.76) and being discharged to inpatient rehabilitation (aOR 1.46, 95% CI 1.10–1.93) (table 2). Regression coefficients for independent variables are presented in table 3. The association between participation in research and both these outcomes did not change with the addition of SU care in the modeling (table 2). Similar results were also seen from the subanalyses of SU care and thrombolysis (table 2). In addition, multivariable sensitivity analysis including patients who were palliated demonstrated similar associations (table e-1 on the Neurology® Web site at Neurology.org). There was no association for independence at time of discharge or for being discharged home or to an aged care facility if participating in research for any of the analyses.

Table 2.

Multivariable results for association between research participation and outcomes of acute hospital care

Table 3.

Regression coefficients from modeling of patient outcomes

DISCUSSION

We report results from a study comparing in-hospital measures of clinical processes and outcomes for patients with stroke who participate in research compared to those who do not. Our results show that patients with stroke who participate in research are not only more likely to receive many recommended processes of care, but are also less likely to die in-hospital compared to nonresearch participants. We also describe results from the national organizational surveys of acute Australian hospitals and found that the majority of stroke research activity is currently focused on the first 48 hours of care and is occurring in hospitals located in urban areas admitting larger numbers of patients with stroke. As the number and type of research trials vary from country to country, results are primarily indicative of research activity and outcomes only in Australian hospitals, and specifically to the stroke population.

In addition to determining the direct effect on the individual patient of participating in research, the other question that is often posed is “Are there better outcomes for patients treated at hospitals that participate in research (regardless if the individual is involved in research) compared with hospitals that are not involved in research?”²¹ While this is an important issue, especially to health care professionals and policy makers, it was not the aim of the current research, which focused on individual benefits for patients from their participation in research activities.

Although measures of clinical processes are potentially more sensitive indicators of quality of care than outcomes,²² few studies have evaluated differences in clinical processes provided to those participating in research compared to nonparticipators. In a previous study, in a population of women with preeclampsia, improvements in only 2 of the 10 processes measured were evident for those involved in research studies.²³ Our findings of improved adherence to processes including SU care, swallow assessment/screen prior to oral intake, timely allied health assessment, and education about important stroke risk factors for research participants has an important consequence, as favorable effects on outcomes have been demonstrated in situations where improved evidence-based stroke interventions are provided.^24–26 It is unsurprising that our results demonstrate that more patients who participated in research received SU care as much of Australia's acute stroke research is undertaken in hospitals with SUs. It is well-established that those admitted to a SU are more likely to receive evidence-based clinical practices and have better survival compared to those receiving only care in general wards.^25,27 However, results from our subanalyses confirmed that our model estimates are robust and remain largely unchanged when SU care or receipt of thrombolysis are controlled for.

The previous review articles present mixed results on outcomes and include studies of varying methodologic strength and quality.^4,6–10 As a result, it can be difficult to determine with confidence if observed differences were due to the effects of participating in the research (trial effect), the clinical interventions (treatment effect), or participant characteristics.¹⁰ In contrast to our outcomes, results from multiple systematic reviews including patients who participated in trials compared to similar patients receiving similar interventions who did not participate in trials concluded there was no evidence of either direct harm or benefit to participation.^7,8,10 Alternatively, results from one review provide evidence of a positive effect on patient outcomes if they are involved in trials.⁴ While the overall combined trial effect from this review was positive, as in our study, authors questioned the transferability of results to other disease states as the majority of these trials occurred in the cancer population. Additionally, the specific outcomes measured in the included studies varied, which makes direct comparison with our results difficult. Nevertheless, the reported overall trial effect was also in line with a smaller review of oncology trials that provides evidence that in this population participation in clinical trials may result in survival benefits over 3 years.⁶ Similarly, published results from recent studies in specific disease areas including multiple sclerosis, HIV, diabetes, and cardiovascular disease have also demonstrated the potential trial effects on patient outcomes.^28–30 While we acknowledge the heterogeneity of studies in this field of inquiry, our results add further evidence of a potential trial effect from the perspective of a stroke population, which has been an area rarely studied before. Overall, the positive results have the potential to influence trial recruitment and possibly reduce dropout rates, especially in open-label trials, when participants hope they will receive a particular intervention.^31,32

In our study, we did not have information on exactly when patients were recruited to studies during their admission, study type (i.e., observational or interventional), or if patients consented and then dropped out. The influence of dropouts could have potentially introduced a response bias, but given the short length of stay (∼6 days) this is unlikely to have had a major influence on our results. Additionally, by only recording a response to being involved in research, our findings of reduced in-hospital mortality among the research participants potentially further suggest a research participation benefit rather than a treatment effect from the intervention.⁴ Previous studies have identified a number of potential sources of trial effect, many of which are applicable to our results. Current results may reflect the effect of both patients and clinicians merely having the knowledge that they are involved in research and being observed (the Hawthorne effect),⁵ or from additional medical reviews, and regular, intensive monitoring of the patient (care effect), which often occurs for research participants.^30,33 Having treatment provided by potentially better informed clinicians involved with research,⁴ or merely from the use of specific guidelines and protocols (protocol effect), which generally are an important aspect of research, may have also been influencing factors.

Trial patients have often been considered a prognostically favorable select group as they generally have a milder form of the disease under investigation than their nonresearch participant counterparts.⁹ Consistent with previous studies, we also found that patients who were younger and more independent prior to stroke were more likely to participate in stroke research.³⁴ Looking at other international stroke trials,^35,36 this may in part be due to selection bias from specific inclusion criteria including premorbid function, and possibly the assumption that patients will survive to the end of the follow-up period. The comprehensive dataset did allow us to adjust our multivariable models for a large number of patient differences and confounders. However, we acknowledge that there were other potential confounding factors that we may not have been able to address, including certain comorbidities, educational level, socioeconomic position, and non-English-speaking background. This means that we are unable to fully attribute causation of results just because of involvement in research. Nevertheless, the results indicate the potential of a positive trial effect and confirm that further research in the area is warranted.

Other limitations relate to the study design and data collection. The retrospective nature of our study only provides a snapshot of care over multiple audit cycles and the influence of missing data is acknowledged. Potential seasonal effects on stroke mortality³⁷ were minimized as data were collected during a 6-month period that covered both winter (55% of data) and spring (45%) months. Understanding why in-hospital mortality differed so markedly between research participants and nonresearch participants in a larger matched cohort of patients would provide further details on the influence of study types, participation, and other factors involved. Overall, having national representation from multiple sites, a large dataset, and the inclusion of organizational data as well as clinical and patient outcome data are strengths of the study.

We found that patients who participate in stroke research receive better care and potentially have improved in-hospital survival outcomes compared to those who do not participate in research. This may be due to factors such as increased monitoring associated with participation in research or greater contact with health professionals. This information may be encouraging to patients with stroke and reduce dropouts in open-label trials.

GLOSSARY

aOR

adjusted odds ratio

CI

confidence interval

ICD-10

International Classification of Diseases–10

LOS

length of stay

NSF

National Stroke Foundation

Q1

25th percentile

Q3

75th percentile

SU

stroke unit

AUTHOR CONTRIBUTIONS

T.P.: drafting of the manuscript, performed the data analyses, and contributed to the interpretation of the data. K.H.: conceptualization and design of the study, manuscript revisions, and interpretation of the data. M.K.: contribution to data analysis methods, revisions, and interpretation of the data. N.A.: contribution to data analysis methods, revisions, and interpretation of the data. D.C.: conceptualization and design of the study, drafting of the manuscript, supervision of analysis, and interpretation of the data.

STUDY FUNDING

No targeted funding reported.

DISCLOSURE

T. Purvis and K. Hill report no disclosures relevant to the manuscript. M. Kilkenny was supported by an Early Career Fellowship from the National Health and Medical Research Council (NHMRC; 1109426). N. Andrew was supported by an Early Career Fellowship from the National Health and Medical Research Council (NHMRC; 1072053). D. Cadilhac was supported by a fellowship from the National Health and Medical Research Council (NHMRC; 1063761 co-funded by National Heart Foundation) and restricted educational grant from Boehringer Ingelheim for work unrelated to this project. Go to Neurology.org for full disclosures.