A pragmatic, multi-centered, stepped wedge, cluster randomized controlled trial pilot of the clinical and cost effectiveness of a complex Stroke Oral healthCare intervention pLan Evaluation II (SOCLE II) compared with usual oral healthcare in stroke wards

Abstract

Background

Patients with stroke-associated pneumonia experience poorer outcomes (increased hospital stays, costs, discharge dependency, and risk of death). High-quality, organized oral healthcare may reduce the incidence of stroke-associated pneumonia and improve oral health and quality of life.

Aims

We piloted a pragmatic, stepped-wedge, cluster randomized controlled trial of clinical and cost effectiveness of enhanced versus usual oral healthcare for people in stroke rehabilitation settings.

Methods

Scottish stroke rehabilitation wards were randomly allocated to stepped time-points for conversion from usual to enhanced oral healthcare. All admissions and nursing staff were eligible for inclusion. We piloted the viability of randomization, intervention, data collection, record linkage procedures, our sample size, screening, and recruitment estimates. The stepped-wedge trial design prevented full blinding of outcome assessors and staff. Predetermined criteria for progression included the validity of enhanced oral healthcare intervention (training, oral healthcare protocol, assessment, equipment), data collection, and stroke-associated pneumonia event rate and relationship between stroke-associated pneumonia and plaque.

Results

We screened 1548/2613 (59%) admissions to four wards, recruiting n = 325 patients and n = 112 nurses. We observed marked between-site diversity in admissions, recruitment populations, stroke-associated pneumonia events (0% to 21%), training, and resource use. No adverse events were reported. Oral healthcare documentation was poor. We found no evidence of a difference in stroke-associated pneumonia between enhanced versus usual oral healthcare (P = 0.62, odds ratio = 0.61, confidence interval: 0.08 to 4.42).

Conclusions

Our stepped-wedge cluster randomized control trial accommodated between-site diversity. The stroke-associated pneumonia event rate did not meet our predetermined progression criteria. We did not meet our predefined progression criteria including the SAP event rate and consequently were unable to establish whether there is a relationship between SAP and plaque. A wide confidence interval did not exclude the possibility that enhanced oral healthcare may result in a benefit or detrimental effect.

Trial Registration

NCT01954212.

Introduction

Pneumonia is reported to impact 6% to 20% of stroke survivors^1,2 and is associated with poorer outcomes, longer hospital stays, greater dependency at discharge, greater healthcare costs, and risk of death.^2–9 Stroke-associated pneumonia (SAP) has been associated with risk of aspiration^4,10 as have stroke severity, alertness, and functional ability,³ yet none fully explain the incidence of SAP.^3,5 The cleanliness of stroke survivors’ mouth and teeth may contribute to the development of SAP.^11,12

Stroke survivors have a higher prevalence of oral Gram-negative bacteria than hospitalized nonstroke peers.¹³ Aspiration of large numbers of such micro-organisms may contribute to the incidence of SAP.^13–15 Dysphagia and poor oral clearance of food and fluid residue further contribute to microbial load and dental decay.¹³ Stroke admissions may have pre-existing oral health problems, such as periodontitis, which is linked to the incidence of cardiovascular disease, diabetes, and possibly stroke.¹³

Organized stroke care, such as protocols of care to manage fever, glucose, and dysphagia,¹⁶ and specialist stroke unit care¹⁷ benefit stroke survivors. High-quality oral healthcare (OHC) may reduce the incidence of SAP and improve patients’ oral health and quality of life,^11,18–21 but definitive randomized controlled trial evidence in a stroke care setting is lacking.

Aim

Working with stroke survivors, carers, and a stroke specialist multidisciplinary group, we co-produced a clinically feasible enhanced OHC intervention.²⁰ In this study, we aimed to pilot the delivery of enhanced OHC across multiple stroke wards in a pragmatic, stepped-wedge, cluster randomized controlled trial of the effectiveness of enhanced versus usual OHC.²³ We considered whether our (i) randomization and blinding procedures were robust; (ii) intervention, data collection, and record linkage processes were viable; (iii) sample size calculations and estimates of screening, recruitment, consent, and retention required refinement; (iv) participants adhered to the intervention; (v) pneumonia event rates could be determined across sites and over time; (vi) there was an association between dental/denture plaque and SAP; and (vii) planned statistical and health economic evaluations were feasible. Our predetermined criteria for progression²² to a Phase III definitive RCT were as follows:

• Viability of the enhanced OHC intervention, data collection processes, and protocol.

• Observation of at least 80% of the anticipated SAP event rate among patient participants during usual OHC. Where rates were 50% to 80%, we would review our recruitment and SAP diagnostic criteria. If less than 50%, we would reconsider our study design.

• Establish that there is a relationship between SAP and plaque.

Methods

Design

We collected usual OHC data (preintervention phase), intervention data (conversion phase), and enhanced OHC data (experimental phase) in a stepped wedge cluster RCT (Supplementary Fig. 1). Using a computer-based randomization sequence and concealment of allocation, we randomly allocated stroke rehabilitation sites to a sequence of fixed, stepped time-points to convert from usual to enhanced OHC over 13 months at each site (Supplementary Fig. 1). Additional trial design and conduct details can be found in Supplementary File.

Participants

All admissions were eligible for inclusion in this pragmatic trial, including all ages, dentition profiles, reason for admission (including nonstroke), cognitive, and communication impairment status. People with incapacity were eligible for inclusion. We sought welfare guardians’ informed consent. Nursing staff (registered nurses, nursing assistants, and student nurses) were eligible for inclusion. We had no patient or staff exclusion criteria. The Scotland A Research Ethics Committee granted ethical approval (13/SS/1304).

Intervention

Our co-produced, multicomponent enhanced OHC intervention sought improvements in stroke survivors’ OHC and outcomes as a consequence of staff delivering enhanced OHC.^20,23 Nurses were encouraged to participate in a 90-minute on-line OHC training course that provided evidence-based or best practice OHC information and a tutorial on OHC assessment and care.²² Assessments and protocols were made available on the ward.²² Staff had access to all necessary OHC equipment, products, and specialist dental services.²² Interventions at patient level were not standardized in this pragmatic trial but tailored by the nursing staff to individuals’ needs by regimen, degree of support, equipment, and products.

Data collection

Outcome data items were collected at patient, staff, and service levels (Supplementary Table 1 including refs.24–28).²⁰ Caldicott Guardian approval granted access to aggregated National Health Services UK data on length of hospital stay and discharge for ward admissions during the study period. We had no other follow-up. Ward-level resource use data were also collected.

Statistical analysis

Analysis was on an intention-to-treat basis. Every effort was made to retrieve missing data. Where outcome data were missing, those records were removed from formal statistical analysis relating to that outcome, unless otherwise specified. The patient-level primary outcome of SAP²⁴ was summarized by study period and site. Statistical analysis compared the incidence of pneumonia between two time periods–before and during enhanced OHC–using a logistic regression generalized linear mixed model,²⁹ which included site as a random effect and date as a fixed effect. The intervention effect was included as a fixed effect using an 0/1 indicator variable to represent the periods before/during enhanced OHC. We did not adjust for any other patient-level covariates. Patient and staff recruitment rates were estimated by the relevant proportion and its 95% confidence interval (CI).

Results

Patients

We enrolled 325 patients of 1548 screened (proportion 21.0% (95% CI: 19.0%–23.1%)). We collected data during usual OHC (135 patients; 105 after stroke), intervention (n = 56; 38 stroke), and enhanced OHC phases (n = 147; 99 stroke) (Supplementary Fig. 2; Supplementary Table 2). Stroke Oral healthCare pLan Evaluation (SOCLE) patients had a median age of 76 (Inter-Quartile Range (IQR): 63 to 83) years and their alertness, diagnosis, capacity to consent, modified Rankin Scale (mRS), dentition, dysphagia, and nutritional status were similar across sites 1 to 3 (Supplementary Tables 2 and 3). Few were considered to lack mental capacity to make their own decisions (n = 51; 15.7%). Site 4 differed with more female participants, more alert, more disabled, incapacitated, with dentures, and a nonstroke diagnosis. Stroke survivors at site 4 (62% of site 4 participants) were a median of 8 (IQR: 0 to 22) days since stroke compared with survivors at sites 1 to 3 (76%–81% of site 1–3 participants) who were admitted at stroke onset (median: 0; IQR: 0 to 1 days). No patients withdrew, dropped out, or reported adverse events. The trial ended on schedule.

Twenty-five participants (7.4%) developed pneumonia²⁴ across the pilot with one site recording the expected rates (n = 19; 21.1%). Physician diagnosis of pneumonia was at a similar rate (8.3%). There was considerable uncertainty regarding the association between the pneumonia event rate and enhanced OHC versus usual OHC (P = 0.62, OR: 0.61; CI: 0.08 to 4.42). Twelve participants contributed data to two phases. One patient contributed across three phases (both usual and enhanced care) but did not develop pneumonia. Only the denominator in summaries and modelling was affected, and thus, we made no adjustment for repeated measures. Dental plaque, denture plaque, and oral health-related quality-of-life scores were similar between usual and enhanced OHC. No adverse events were reported. Based on aggregated routinely collected data of 2613 ward admissions, 10 SOCLE participants died (of 238 across sites) in three cases following pneumonia within 3 months of admission (Supplementary Table 4).

Nursing staff

We recruited 112 nursing staff of 123 employed (proportion 91% (95% CI: 85% to 95%)) who contributed data to the usual OHC (n = 108), intervention (n = 74), and enhanced OHC phases (n = 83). They included registered nurses (n = 62, 55%), assistants (n = 44, 39%), students (n = 5, 5%), and one was unreported. Staff (n = 74) demonstrated no changes in their OHC knowledge or OHC attitudes after training, nor did the registered nurses differ from other nursing staff in their scores (Supplementary Fig. 3). Documentation of OHC assessment and care plans was poor but increased slightly during the enhanced OHC phase (OHC findings: usual care = 18/244 (7.4%), enhanced care = 32/302 (10%), OHC plan usual care = 6/244 (2.5%) and 9/302 (3.0%)) particularly across sites 2–4).

In total, 37 registered general nurses (RGN) and 37 clinical support workers (CSW) completed the training across the four sites. The assumed RGN salary was a Band 6 (£44 per hour) and CSW salary was a Band 2 (£24 per hour).³⁰ The median (1 h and 2 min) and average time taken to complete (2 h and 17 min) showed the range of training costs. Cost analysis indicated total staff training costs of between a median of £446 to £823 per site (£2600 across all sites) and mean of £986 to £1818 per site (£5745 across all sites).

Service level processes of care

Ten specialist dental referrals were made during the trial and six during the usual OHC phase. Three sites increased their monthly average delivery of denture cups, saliva spray, toothpaste and Nystatin, and mouthwash during enhanced OHC (Supplementary Table 5). Resource use for equipment and products was calculated for each site using reported information and NHS procurement costs. During the usual care the average monthly cost across four sites was £268 and, during the enhanced OHC phase, the average monthly cost was £248.

Discussion

We considered the strengths and limitations of this study in the context of our predetermined principal research questions and pilot trial progression criteria.²²

Randomization and blinding procedures

Generation of our randomization sequence was robust and our concealment of allocation process was adhered to. We acknowledge the blinding limitations in the context of a stepped-wedge cluster RCT design. Future studies could ask researchers to indicate which phase data collection contributed to as a measure of blinding success. Training research staff to collect data while withholding key information about the trial design may be possible. We chose not to do this.

Viability of intervention, data collection, and record linkage processes

Our intervention was viable. Assessment, protocols, and training were facilitated by senior nurses. Through site engagement, we ensured availability of dental equipment, products, and specialist services. Data collection processes were feasible though record linkage took considerable effort and required additional permissions. Data collection during a full trial may require modification given the increasing use of electronic medical records though the recent general data protection regulation³¹ may facilitate such investigations in future. Shared responsibility across multiple part-time recruiters, coupled with rapid discharge resulted in many patients going unscreened and may have had implications for patient recruitment.³²

Estimates of screening, recruitment, consent and retention, and adherence to the intervention

The routine health service data provided accurate rates of patient flow through the ward, providing vital insight to screening, recruitment, and consent rates.²² Few participants contributed data to more than one phase and there was an absence of patient drop-out from the trial. Few severely impaired, unconscious, incapacitated participants participated in this trial. Gatekeeper bias or recruitment logistics for this clinically relevant patient subgroup may have impacted on recruitment profiles.³³ Strategies to support the recruitment of this clinically important subgroup to future trials might include greater availability of recruiters, alternatives to face-to-face information provision and consenting, and better education of research staff.

We observed poor staff adherence to the documentation of OHC assessments and plans. A change from paper to electronic records at two sites during the trial (the OHC assessment record was reduced to a tick box and there was no place to record an OHC plan) hampered documentation. Adherence to the optional training component varied by site–between 80% and 94.7% of staff at three sites participated in the training. In contrast, less than half the staff (41.1%) in the fourth site participated (Supplementary Fig. 2).

Pneumonia event rates and other between-site diversity

Pneumonia event rates differed markedly across sites with the overall event rate (7.4%) lower than previous suggestions.² Recent systematic review evidence suggested an event rate of 6–10% is potentially a more accurate multisite expectation.^1,34 Site 4 dominated the observed pneumonia rate, though there were fewer deaths among these participants and thus may suggest a selection effect, where very unwell patients at emergency admission were not well enough to transfer to that site. As a population however, the participants from the site were more severely affected by stroke, incapacitated, and wore dentures than participants at other sites. The limited incidence of SAP amongst study participants prevented further exploration of the association with plaque (dental/denture).

We observed other marked differences between sites including screening to recruitment conversion rates, equipment use, and staff participation. More site 4 staff declined to participate in the study, but few dropped out after recruitment. In contrast, many site 2 staff agreed to participate but exited the trial pretraining (n = 20) leaving just 38% to complete training (13/34). Pre-existing access to specialist OHC products and dental health professionals varied by site; site 2 used fewer toothbrushes, denture cleaner, and toothpaste; and site 4 had voluntary input from specialist dentist as required.

Modified sample size calculations for a Phase III trial

We recruited fewer than estimated patient participants (325–400 anticipated) and almost double the estimated staff participants (112 compared with 60 anticipated). The low pneumonia rate meant that we did not reach our predetermined rate of 80% of the anticipated event rate (20%). We reviewed the adjustment of the pneumonia diagnostic criteria but observed little difference on the event rate (7.4% on primary outcome criteria; 8.3% through physician diagnosis). Through routine data linkage, we know that 58 nonparticipants died following pneumonia. The high intracluster correlation (Supplementary Fig. 3; ICC = 0.41) provides useful information for planning future studies. Due to the low incidence of pneumonia among SOCLE patients, calculation of the ICC CI was impeded by the sparse pneumonia counts across sites (Supplementary Table 4).

Our SOCLE pilot is the largest multicentered trial of an OHC intervention delivered to patients and staff in typical stroke rehabilitation wards to date comparing enhanced OHC provided by nurses to usual OHC and its impact on SAP. The wide CI did not exclude the possibility that enhanced OHC may result in a large benefit or detrimental effect. Our pilot pragmatic trial lacked power to draw firm conclusions about our pneumonia outcome (this was not our primary objective). The clinical implications are few, though the range of pneumonia event rates across sites is noteworthy and may highlight the nature of clinical diversity across wards perhaps because of different patient populations at different sites.

Conclusion

Our SOCLE pilot trial findings showed that a large-scale stepped-wedge cluster trial to evaluate the efficacy and effectiveness of a complex OHC intervention versus standard NHS care for oral health after stroke was feasible and accommodated marked between-site diversity. Electronic records should support documentation of OHC assessments and plans. We did not meet our predefined progression criteria including the SAP event rate and consequently were unable to establish whether there is a relationship between SAP and plaque. Future trials should base SAP expectations on recent multisite incidence data and consensus definitions of SAP.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The SOCLE II trial was funded by the Stroke Association, UK (TSA 2012/09). MB, AP and the NMAHP Research Unit are funded by the Chief Scientist Office part of the Scottish Government Health and Social Care Directorates. The views expressed here are those of the authors and not necessarily those of the funders. The authors would like to thank the participants (patients and staff), the principal investigators, research staff and members of the joint steering and data monitoring committee for their input to this pilot trial.

Supplemental material

Supplemental material for this article is available online.