Abstract
Objectives
To identify the critical dimensions of hospital Chief Executive Officers’ (CEOs) involvement in a quality and safety initiative and to offer practical guidance to assist CEOs to fulfil their leadership role in quality improvement (QI).
Design
Qualitative interview study.
Setting
20 organisations participating in the main phase of the Safer Patients Initiative (SPI) programme across the UK.
Participants
17 CEOs overseeing 19 organisations participating in the main phase of the SPI programme and 36 staff (20 workstream leads, 10 coordinators and 6 managers) involved in SPI across all 20 participating organisations.
Main outcome measure
Self-reported perceptions of CEOs on their contribution and involvement within the SPI programme, supplemented by staff peer-reports.
Results
The CEOs recognised the importance of their part in the SPI programme and gave detailed accounts of the perceived value that their involvement had brought at all stages of the process. In exploring the parts played by the CEOs, five dimensions were identified: (1) resource provision; (2) staff motivation and engagement; (3) commitment and support; (4) monitoring progress and (5) embedding programme elements. Staff reports confirmed these dimensions; however, the weighting of the dimensions differed. The findings stress the importance of particular actions of support and monitoring such as constant communication through leadership walk rounds and reviewing programme progress and its related clinical outcomes at Board meetings.
Conclusions
This study addressed the call for more research-informed practical guidance on the role of senior management in QI initiatives. The findings show that the CEOs provided key participation considered to significantly contribute towards the SPI programme. CEOs and staff identified a number of clear and consistent themes essential to organisation safety improvement. Queries raised include the tangible benefits of executive involvement in changing structures and embedding for sustainability and the practical steps to creating the ‘right’ environment for QI.
Keywords: Health Services Administration & Management, Qualitative Research
Article summary.
Article focus
To qualitatively identify the perceived critical dimensions of hospital Chief Executive Officers (CEOs) involvement in a quality and safety initiative: the Safer Patients Initiative (SPI).
Key messages
The findings show that the CEOs provided key participation that they and others considered to significantly contribute towards the SPI programme.
Five primary managerial roles within the SPI programme were identified: (1) resource provision; (2) staff motivation and engagement; (3) commitment and support; (4) monitoring progress and (5) embedding programme elements.
Queries raised include the tangible benefits of executive involvement in changing structures and embedding for sustainability and the practical steps to creating the ‘right’ environment for quality improvement (QI).
Strengths and limitations of this study
This study addresses the call for more research-informed practical guidance on the role of senior management in QI initiatives. It makes an evidence-based contribution to the quality debate around leadership in healthcare by drawing on original empirical material collected across 20 UK healthcare settings. The findings impart guidance for other managers at this level opting into a similar intervention and outline certain actions pertaining to different stages of the programme.
The CEOs’ self-reports may be subject to social desirability bias. Similarly, self-selecting bias may derive from the fact that the CEOs volunteered for the high-profile initiative, arguably leading to an overestimation of the involvement that senior managers at this level would typically engage in within most improvement initiatives within their Trusts. However, we have tried to lessen this limitation with supplementary analysis of staff views.
No association can be made between the CEOs’ dimensions and the successes/failures of the SPI programme.
Introduction
The number of quality improvement (QI) initiatives in the healthcare sector is growing rapidly. Their aim is to improve processes, structures and systems through continuous QI techniques in order to improve outcomes of care.1–3 Research examining these programmes and larger-scale collaboratives have found some evidence of their impact;4 their sustainability5 6 and economic benefits.7–9
Effective support from senior managers is believed to be critical to the success of their programmes.10–12 In a review of healthcare Board level and senior management behaviours associated with QI outcomes, Øvretveit13 identified a plethora of studies that impart the importance of managerial involvement and engagement in quality and safety improvement. Actions frequently referenced as beneficial included displays of senior management commitment and support14 and creating the right culture.15 However, Øvretveit13 concluded that there is little research-based practical guidance to outline the details of the senior management role in leading improvement and called for more academic research on this topic. This study addressed the issue by exploring the self-reported participation of Chief Executive Officers (CEOs) involved in the second phase of an organisation-wide quality and safety collaborative, the Safer Patients Initiative (SPI), to better understand the role of Board level senior managers within such initiatives.
SPI and our previous research
Funded by the UK Health Foundation, the SPI was developed by the Institute for Healthcare Improvement (IHI). It was piloted with four UK National Health Service (NHS) organisations in its first phase (2004–2006) and applied at a further 20 in its second phase (2006–2008).16 17 Designed to achieve improvements in patient safety, SPI attempted to make changes at an organisational level and in front-line care processes within four clinical areas through implementing a number of clinical working practices with continuous QI and process measurement techniques. The main elements of the SPI programme are outlined below in box 1. Today, many of the principles of SPI have continued with 18 of the involved organisations opting in to the follow-up initiative ‘The Safer Patients Network’.
Box 1. The Safer Patients Initiative—a description.
SPI aims
Mortality: 15% reduction
Adverse events: 30% reduction
Ventilator-associated pneumonia: 0 or 300 days between
Central line bloodstream infection: 0 or 300 days between
Blood sugars within range (intensive care): 80% or more within range
MRSA bloodstream infection: 50% reduction
Crash calls: 30% reduction
Harm from anticoagulation: 50% reduction in adverse events
Surgical site infections: 50% reduction
Workstreams (example change elements)
Perioperative care (deep-vein thrombosis prophylaxis, β-blocker use) reduction
Medicines management (medicines reconciliation, anticoagulants) reduction
General ward care (early warning systems, rapid response team and hand hygiene) reduction
Critical care (ventilator bundle, central line bundle and daily goal sheets) reduction
Leadership (leadership walk rounds, strategic prioritisation of quality and safety) reduction
Programme tools and methodology
Continuous quality improvement: semiautonomous teams
PDSA cycles and small tests of change
Incremental spread to successively larger work systems
Process measurement and analysis of run charts to determine effects
Expert faculty support from Institute for Healthcare Improvement (site visits, conference calls and online email support) reduction
Large-scale learning sessions for multidisciplinary improvement teams
Online extranet for uploading and comparing process data with monthly feedback
Collaborative learning community for networking and sharing best practices
In our previous research, we have investigated a number of factors affecting the SPI programme. These include organisational readiness for SPI, clinicians’ engagement with SPI, leadership walk rounds prescribed by SPI and predictors and perceptions of impact of SPI. In the pilot phase of SPI, survey responses by those involved (clinical leads, coordinators and management) rated senior management support as the highest-ranking strength in the implementation of SPI.18 Additional qualitative analyses revealed manager involvement as a reported facilitator of medical engagement in SPI.19 This involvement comprised allocating resources, having good management–doctor relationships and commitment at executive management level. Other interview findings showed that senior managers helped to remove barriers and empower staff to change processes through events such as leadership walk rounds.20 In research on the main phase of SPI, we extracted further perspectives on leadership walk rounds that revealed that they can help executives learn about their organisations and help clinical staff overcome misperceptions of the executives.21
In our longitudinal quantitative work, programme implementation factors, including senior management processes, were found to contribute significantly to change in organisational safety climate and capability linked to programme milestones, above and beyond the effects of programme contextual factors and organisational preconditions.22 We have not previously identified which senior management behaviours are perceived to be important. In other investigation across two time points, we identified strategies for sustaining SPI that were reported to require senior management help on financial and human resources for the programme, as well as incorporating patient safety into induction and training.23 In addition, the coordinators considered ‘management involvement’ generally to facilitate continuation of the programme and suggested that it was essential to feedback to senior management to keep SPI aims high on their agendas to improve their understanding and enthusiasm for the programme. Exploring CEO actions may highlight the reasons why this is important, for example whether feedback elicited follow-up actions by the managers. Other generic findings from investigation at the main phase revealed executive management commitment to quality as a strength of the programme according to ratings from both senior management and frontline staff.24
In summary, our previous research has suggested an importance in managerial involvement and commitment in SPI and identified some potential dimensions of this involvement. However, these have not been described in detail or confirmed by CEOs directly. Our specific research aims are to identify the critical dimensions of hospital CEOs involvement in SPI, and to offer practical guidance and classifications that will assist CEOs to fulfil their leadership role in QI.
Methods
Sample
Setting
Interviews were carried out across all 20 NHS hospitals participating in the second phase of the SPI programme across four geographical locations in the UK: England, Northern Ireland, Scotland and Wales. The hospitals varied in terms of type (eg, teaching) and size. The biggest participating Trust had a total of 22 000 staff (not all of their hospitals were involved in SPI) and the smallest had 2100 staff (est. June 2008). Two Trusts each had two hospitals involved in SPI.
Participants
A purposive sampling strategy across all 20 organisations aimed to include the CEOs at all of the participating organisations. These senior managers were often involved in the ‘Leadership workstream’ that governed the SPI programme across all the clinical workstreams in which it was implemented. This workstream were advised to walk round the hospital in ‘Leadership Walk rounds’ and to have a strategic prioritisation of quality and safety.
Seventeen interviews were conducted with CEOs representing 19 of the 20 hospitals participating in the SPI programme. There were only 17 participants because one CEO did not participate in the interviews (we have reason to believe this was because he/she was busy in the process of moving on to another Trust), and two of the CEOs managed more than one participating hospital. Specifically, every Trust was managed by a different CEO and two Trusts had two hospitals participating in the SPI programme. Please see table 1 for participant demographics.
Table 1.
Gender | Clinical/non-clinical background | Tenure in Trust | Number of SPI hospitals overseen by CEO |
---|---|---|---|
Male | Non-clinical | 6–9 years | 1 |
Male | Non-clinical | 0–11 months | 1 |
Female | Clinical | 21 or more years | 1 |
Male | Non-clinical | 3–5 years | 1 |
Male | Non-clinical | 1–2 years | 1 |
Female | Non-clinical | 1–2 years | 2 |
Male | Non-clinical | 6–9 years | 1 |
Male | Non-clinical | 0–11 months | 1 |
Male | Non-clinical | 3–5 years | 1 |
Female | Non-clinical | 10–20 years | 1 |
Female | Non-clinical | 10–20 years | 1 |
Male | Non-clinical | 6–9 years | 1 |
Male | Non-clinical | 0–11 months | 1 |
Female | Clinical | 0–11 months | 1 |
Male | Non-clinical | 1–2 years | 2 |
Male | Non-clinical | 10–20 years | 1 |
Male | Non-clinical | 3–5 years | 1 |
CEO, Chief Executive Officer; SPI, Safer Patients Initiative.
Supplementary analysis was carried out on 36 interviews with staff involved in SPI to verify/challenge the CEO self-reports. This comprised 20 workstream clinical leads (5 per workstream), 10 programme coordinators and 6 management (2 directors of nursing, 2 medical directors, a general manager and a clinical governance manager), which amounted to two interviewees per CEO, including the CEO not interviewed.
Procedure
The data collection period was between April and August 2008 towards the official end of the SPI programme and comprised face-to-face interviews lasting approximately between 45 and 60 min.
Interviewees were shown a research information sheet, briefed on their anonymity and asked to sign a form consenting to audio recording the interviews for transcription and analysis. A standardised semistructured interview topic schedule was used by two interviewers (pairings of five different researchers, JB, AP, SB, SI and APo), which addressed the senior managerial role along with a host of issues regarding the programme. This is because, as shown in the introduction, the study investigated a number of issues surrounding SPI of which the senior management role was one topic of investigation. Example questions directly asking CEOs about their role included: “What are your main responsibilities?” and “How were/are you involved in SPI?” Staff were asked “How was/is your senior management/executives involved in SPI?”
Data analysis
The interviews were transcribed by professional transcribers. Qualitative analysis was performed, based on inductive grounded theory analysis techniques of open coding, constant comparative analysis and theory building, with the aid of NVivo 8 software.25 26 The 17 CEO transcripts were divided and independently coded by the five researcher interviewers (JB, SB, SI, AP and APo). This comprised identifying any text, indirect or direct, pertaining to the executives’ involvement (actions, work or contributions) within the SPI programme. This resulted in one code containing all references to CEOs involvement. Line-by-line open coding was then performed by one researcher (AP) on all of the CEO transcripts to deconstruct the dataset and draw out singular dimensions. This was also carried out on this node coded by the other researchers to compare inclusions. At this stage of analysis, highly specific codes related to perceptions of CEO contributions and actions were identified. The importance of their involvement in the SPI programme, and barriers and enablers were also coded to provide additional contextual information to the managers’ roles. All references coded concerned the managers’ actual involvement/contributions and barriers or enablers faced, as opposed to their opinions on what managers in their position should do or would likely face. The constant comparative method was used to compare emerging codes with earlier codes drawn from the dataset and individual codes were grouped into related themes in order to build a model of the main dimensions and their subdimensions. No previous theory was used to analyse the data, all categories were developed from the data. After iterative refinement of the relationships, a model was identified that consisted of the critical dimensions of the CEOs involvement within the SPI programme, based on the CEOs’ reports. To ensure reliability of coding and interpretation, a sample of data fragments were checked and resolved through dialogue with other members of the team by one researcher (AP) identifying differences in coding between the five coders and speaking with the coders in question to arrive at an agreement. The model was considered by external members of the team for their opinion on whether the subdimensions have face validity under the chosen dimensions. The same analysis was carried out on staff transcripts. The dimensions from the staff reports were compared with the model that emerged from the self-reports. The findings section pertains to the CEO reports, with a supplementary summary of the reports by staff.
Findings
The levels of involvement in the programme varied between the executives; however, all gave accounts of the value that they believed to have brought at all stages of the process. They considered their involvement in the initiative as a significant influence on the potential for programme success/failure.
I went away on leave, came back, and it had just all gone downhill because I wasn't there. (Interviewee 8)
The most reported barrier to their involvement was their time constraints to participate within programme efforts, which was often attributed to the demands of managing a large Trust. Facilitators of their engagement included early involvement in the process (from helping at the application stage or/and from attending the first learning session), learning about the programme (such as the QI techniques, the targets set, the support networks available and the motivational impetus delivered by IHI), and having other executives and staff engaged with the programme.
it's really important the Board is engaged early on in a real way and that the Board begins to see the data. (Interviewee 3)
Five primary managerial roles within the SPI programme were identified (presented in table 2). These dimensions are described within this section along with example quotations provided in table 3. In terms of weighting, the dimensions ‘commitment and support’ and ‘monitoring progress’ were referred to by almost all CEOs. Most CEOs also discussed ‘embedding programme elements’ and ‘staff motivation and engagement’. Resource provision was mentioned less than the others, but was still referenced by well over more than half of the CEOs. Although not discrete from one another, our findings show some indication of the stages in which CEOs most get involved in these dimensions, most notably resource allocation before the start and (to a lesser extent) at the end of the programme, followed by engagement, motivation, commitment and support for staff and towards the end of the process the CEOs are more likely to engage in decisions and strategies to embed the programme elements in order to sustain it.
Table 2.
First order dimension | Subdimension | Dimension description |
---|---|---|
1 Resource provision | 1.1 Securing funding | This factor refers to the CEO function of securing funding for the SPI programme and allocating financial and human resources to aid the implementation and continuation of the programme |
1.2 Resource allocation | ||
2 Staff motivation and engagement | 2.1 Motivation and empowerment of staff | This factor describes CEOs motivating, involving and engaging clinical staff with the SPI programme through communication, methods of empowerment and reinforcement |
2.2 Shared dialogue | ||
2.3 Reinforcement of staff involvement | ||
3 Commitment and support | 3.1 Display of visible commitment | This factor refers to the CEOs’ demonstration of their own commitment to the programme along with the CEOs’ role of support (not through resources) to clinical staff involved in SPI. This includes ‘creating the right environment’ for staff and ‘selling’ the programme to them |
3.2 Creation of right environment/climate | ||
3.3 Directing staff and stating purpose | ||
4 Monitoring progress | 4.1 Reviewing SPI measures | This factor illustrates the CEO activity of monitoring programme outcome measures and regularly requesting and reviewing overall performance on SPI, as well as indirectly generating accountability on progress |
4.2 Performance management | ||
5 Embedding programme elements | 5.1 Strategy and agenda change | This factor comprises of changes made by the CEOs to strategies, agendas and processes in order to integrate SPI procedures and practices into them, so that they are sustained |
5.2 Structure change and embedding for sustainability |
CEO, Chief Executive Officer; SPI, Safer Patients Initiative.
Table 3.
First order dimension | Subdimension | Example quotes |
---|---|---|
1 Resource provision | 1.1 Securing funding | “we would probably take a paper to our Trust executive group shortly after that [the end of IHI involvement in the programme] with a decision…whether to continue on the current method [SPI approach], if so, are we going to internally fund it” (Interviewee 6) “We did make a decision to put aside a £200000 patient safety reserve, a SPI reserve if you like, to fund the consequences of any initiatives that might come out or any requirements that might come out.” (Interviewee 7) |
1.2 Resource allocation | “we resourced the central office, if you want to call it that, and tried to ensure that people had time, and energy, and the desire to do the right thing there.” (Interviewee 16) “You have to do it and do it well and do it properly and fully and resource it properly. And I guess the NHS as a whole and to some extent us as well have a history of getting in to projects, not resourcing them properly, and then doing them half heartedly. And then they never work and you wonder why, and the answer's bloody obvious actually. But they won't let you do that with SPI.”(Interviewee 12) |
|
2 Staff motivation and engagement | 2.1 Motivation and empowerment of staff | “I think we created the appetite. Nobody was knocking on our door saying they wanted to do patient safety so we created the appetite. So I guess that was top down.” (Interviewee 9) “we've slowly over time ..[delegated work].. to try and increase level of autonomy..So I suppose it was part of me trying to free up people's thinking actually..my first couple of meetings saying, well what [is] 8 of those at 300 quid? Well do it you know and they just found that really liberating because that meant they made some really big strides in the middle of the project.” (Interviewee 14) |
2.2 Shared dialogue | “what I see it [my role] as doing is setting an example that's about having the right dialogue.. And once you've got that engagement, and you've got that dialogue, these issues become central to the debate.” (Interviewee 16) “talking to the staff actually and more importantly listening to the staff about what's going on. You always learn such a lot..When did you last have an incident? What was, what caused it? What did you do about it?.. How many opportunities do you get to raise these sorts of issues?” (Interviewee 13) “They [walk rounds] help the visibility mantra which everybody says about executive teams don't they? They have been an interesting cross check about the things that you think are going on in the organisation” (Inteviewee 17) |
|
2.3 Reinforcement of staff involvement | “clearly if they've [clinical staff] not been following our policies in terms of hand washing and so on, they'll be disciplined. Simple as that..I've got nurses ringing me up saying I've told a doctor off, he hasn't changed his behaviour and we're now following that up..They've been talked to..some of that is about saying, excuse me, but you are doing this actually.” (Interviewee 3) “what I then used..saying right where are all the surgical CDs [Clinical Directors] who are looking at their shoes, why aren't you doing it? And next time we meet to talk about this I want to know your experiences on how you do it, so you sort of try and create a purpose to it” (Interviewee 14) “initially it was more around initial conversation with [director name] and getting him on Board” (Interviewee 16) |
|
3 Commitment and support | 3.1 Display of visible commitment | “If they don't see you believe in it [SPI], why the hell should they struggle?” (Interviewee 2) “I think the most important role is to be seen to be committed to it [SPI].. It's all very well being a figurehead, but this doesn't allow you to get away with just turning up for the celebratory glass of wine or whatever it is. You've actually got to be in there and do it”(Interviewee 12) “we've puffed our chests up and said we are serious about this and then we have to follow through. But what's interesting now that we are following through, people believe it and there is a visible, noticeable difference in the last two or three weeks out there on the wards in terms of consultants, they're taking their ties off, they're rolling their shirts up, they're washing their hands and people are challenging.” (Interviewee 3) |
3.2 Creating the right environment/climate | “What a Chief Executive has to do is to build a coalition of support to a broad framework within which people work.” (Interviewee 15) “And it's about creating the right climate..in some respects I created a climate of restraint” (Interviewee 14) |
|
3.3 Directing staff and stating purpose | “one of the things I was keen that we did was to make this something that the whole Board was interested in and not just the acute hospital because some of the learning will run across other parts of our service out in the community. So from day one we put together a very broad communication.” (Interviewee 9) “we have a five year vision that actually can be brought down to one sheet of paper. Eventually it will be in several vehicles, it will be a glossy document that will be presented to all new staff, that will be brought out at the start of any project meeting...on the one page one, the work SPI appears..So a Chief Executive has to do some top down things, about setting a tone, setting a direction...The first one [task], [is] to adopt it [SPI], to take advice, to accept advice. The second one, then, is to learn enough about it that you can speak authoratively. Chief Executives have to be able to speak about everything for 90 seconds..so a Chief Executive needs to have a 90 second elevator speech..that you can turn to a group of doctors, in the right situation, and say SPI is really the thing because, and then you list whatever” (Interviewee 15) |
|
4 Monitoring progress | 4.1 Reviewing SPI measures | “we are seeing well populated Run Charts, we're being able to use and understand the data more effectively, both at a senior level and within the teams.” (Interviewee 9) “I'm regularly looking at the information that is produced from it [SPI], I wouldn't say I'm looking at the data itself...It's normally a presentation, or patient story, or something like that..so that's changed the Board [agenda] in that you're not straight into finance..But whether we're hugely different to where we were 18 months ago, I don't know really.”(Interviewee 10) “at the breakfast meetings..we go through all the [SPI] measures” (Interviewee 7) |
4.2 Performance management |
“we've got a different design for our performance management.. data points that will be demonstrated for assurance purposes at the Board.” (Interviewee 3) “I think it's [SPI is] in our operational plan, it's a performance measure in there, so therefore, when we meet the divisions on a monthly basis, one of the things we'll be asking them for is their SPI measures.” (Interviewee 10) |
|
5 Embedding programme elements | 5.1 Strategy and agenda change | “for me, it's, it'll [SPI will] be a way of doing things, integrated into where we are, and it has to be key item on every agenda, the things that's shaping the debate.” (Interviewee 16) “I had to make some clear statements from the word go about where it [SPI] was on the agenda, so it was, it has been the first item on the Management Board agenda for the last 18 months. The patient SPI, right, where are we, what have we achieved, what are we doing?..we've set, tried to set it in the strategic context of what the Trust is doing. The Trust Board adopted a new mission statement..that there would be three main themes..and one of them was the Safer Patient Initiative and patient safety.” (Interviewee 13) |
5.2 Structure change and embedding for sustainability | “[we need to] make sure that the elements of SPI that we keep are integrated into our performance management regime.” (Interviewee 4) “the way we've rolled out SPI..we integrated it into people's directorate objectives, that's why we keep the profile up.” (Interviewee 5) “that's how you begin..you narrow the gap between the activities of the initiative and disciplines around directorate management and delivery, you narrow that by drawing it together and holding people to account for outcomes” (Interviewee 14) |
CEO, Chief Executive Officer; SPI, Safer Patients Initiative.
Resource provision
Funding to support the SPI programme was deemed important and many CEOs recognised this as one of their primary contributions to the programme. This took two forms: their activities to bid and secure funding from outside the Trust (both at the application stage of SPI and for its continuation) and their authorisation of internal Trust resources (both financial and human resources). Each organisation involved in the programme were provided with an allotted sum of money (approximately £270 000 per hospital) and external resources, such as external monitoring by IHI. After the official 2-year period of implementation, withdrawal of these resources instigated plans to ensure that resources covered by initial funding and support could be continued. The most common resources authorised by CEOs for the SPI programme were: time allowed for SPI work and training; data collection and data support personnel; and an SPI coordinator to oversee the project.
Staff motivation and engagement
The CEOs described activities that empowered, motivated and reinforced staff involvement with the SPI programme. In accounts of motivating staff, the CEOs described ‘creating an appetite’ and ‘free[ing] up peoples thinking’, reporting an aim of changing staff attitudes towards the programme. Their actions to empower staff also included allowing them more power to authorise resources. Leadership walk rounds were considered as a particularly useful tool for shared dialogue and as a listening exercise. The walk round involved speaking with frontline staff across the hospital and was the principal activity of the CEOs position in the ‘leadership workstream’. Constant communication with staff was critical to encourage their engagement with the programme. At times the CEOs were called in to deal with resistance to the programme, whereby they would either discuss the situation with the resisters, attempt to instill a sense of purpose, or in the worst case, threaten disciplinary measures for not adhering to SPI practices. Doctors were singled out as the profession with the most resistors, therefore facilitating doctor engagement was a commonly cited role. CEOs who attended SPI learning sessions to learn about relevant improvement practices reported that their learning helped when engaging staff, as they were more knowledgeable on various aspects of the programme, such as QI techniques and targets set.
Another critical task was encouraging Board buy-in through highlighting the programme strategies and targets. An NHS Board is made up of a chairman, executives, directors and non-executives and, through regular meetings they jointly oversee, offer direction and are responsible for the financial and quality performance of the hospitals within their Trust. Employed by the Trust, the full-time executives/directors (eg, CEO and Medical Director) are responsible for the day-to-day oversight of the hospitals and together with the chair and non-executives (recruited externally to the Trust on a part time basis) are all responsible for overall governance, strategy, achieving performance targets and standards. Therefore, collectively they hold influence over the quality and safety of their organisations.
Commitment and support
All 17 CEOs highlighted the importance of personal commitment and most believed that they acted as a support to staff implementing the programme. Some CEOs described acting as a role model to others and most agreed on the powerful effects that their visible commitment had. Demonstrations of commitment included: attending learning sessions; emphasising the purpose of SPI; attending leadership walk rounds; integrations of safety into the Board agenda such as safety stories at meetings and prioritising it on the agenda; speaking at sessions to explain the programme; and providing approval for SPI-related practices. These were considered demonstrations of commitment to SPI because they required observable effort by the CEOs to prioritise, promote and become involved in the programme. Some made the point that acting as a figurehead is not enough, and that visible acts of commitment need to follow. A few described the potential for loss of momentum if their commitment was absent. Several interviewees recognised their role in creating the right climate and environment for others to undertake the programme work effectively; however, they fell short of offering detailed description of what this actually involved. The interviewees reported to further aid their staff with statements of purpose and direction. This endeavour was also referred to as ‘selling’ the process. This was done through disseminating the programme aims and targets via workshops to staff and presentations to the Board. The CEOs also increased their involvement when SPI work activity was not heading in the right direction.
Monitoring
Monitoring the progress of the initiative was a frequently reported activity. The CEOs monitored progress by reviewing SPI outcome measures, reading reports, checking information and asking for information on particular programme actions and challenges at Board meetings. Outcomes were reviewed on a weekly or quarterly basis depending on the Trust, often in the form of presentations, safety-style dashboards and Run Charts.23 While regularly reviewed, it was not always analysed or actioned; however, many CEOs agreed that it both raised awareness and flagged safety issues, as well as offering the Board an opportunity to prioritise, openly discuss, understand and address trouble areas. Monitoring of progress was not only to explore challenges, but also as a way of ensuring that targets were met. Feedback to senior management at Board/project meetings on whether staff were complying with SPI prescribed activities was thought to be a powerful influence on staff engagement and accountability. This is because staff were influenced by positive or negative responses from senior management. Accountability was generated at these meetings through assessment of targets met and actions delivered. The CEOs primary intention to monitor the process and its key clinical indicators was to become familiar with the programme and to keep track of progress rather than to improve compliance. Timeframes were set by the workstream leads and coordinators but CEOs would query the programme leads if they were falling behind on self-imposed deadlines and targets. Outside of the meetings, the CEOs did not audit the programme's progress or compliance to it; instead they relied on the implementers of the programme to report back on these, especially if there were any problems.
Embedding programme elements
Many CEOs discussed changing system processes and strategies in order to facilitate change necessary for new SPI activity and procedures. Embedding them into existing systems and processes was considered the most efficient way to sustain practices and the most cited approach used. The profile of quality and safety targets and plans were raised through adding SPI objectives high on the agenda and amending strategies to focus on SPI prescribed activity and aims. Examples included adding SPI targets into mission statements and strategic objectives. Integration of programme elements into existing systems involved amendments to processes, such as changes to performance management systems and strengthening lines of accountability associated with targeted outcomes. Putting reporting mechanisms in place and incorporating SPI elements into other existing initiatives, such as Lean (a continual improvement process model), were other frequently quoted methods of integration, as was including practices into staff objectives and individual performance management.
Staff reports of dimensions of CEO involvement in SPI
Overall, the reports from the clinical workstream leads, programme coordinators and other managers involved in the SPI programme confirmed that executive involvement in the programme was important. The dimensions of CEO involvement can be closely matched to those that emerged from the self-reports (please see table 4 for example quotations) However, different weightings were placed on the dimensions to those offered by the CEOs’ transcripts and two subdimensions were not confirmed. The most referenced dimension in the staff reports was of ‘commitment and support’, followed by the majority referencing ‘monitoring progress’ and over half reporting ‘staff motivation and engagement’. ‘Resource provision’ was mentioned by only a quarter of the interviewees almost solely referring to allocation of resources (ie, data collection, IT help and backfill time) rather than securing funding. Even fewer mentioned the action and benefits of the CEOs embedding programme elements, with no mention of their activities to change structures and embed programme elements for sustainability, instead mentions were of agenda change alone. No new dimensions emerged from the staff data.
Table 4.
First order dimension | Example quotes |
---|---|
1 Resource provision | “Any other support [from Board and CEO] has been around trying to acquire resources, so for instance there's a large infection control component and .. we've had a nurse on this site who's been collecting information around central lines, VAPs and so on and they haven't had that resource on the other site, because we were two separate Trusts. So they collected their data on VAPs and other infections in a different way. Because we're one Trust now and we're taking this forward, we want to have the same process on all the sites, so that's where the management are essential, so it's that sort of financial and resource support” (Trust 12, clinical lead, critical care) “some of the changes that we've needed with IT and that I have pushed up to the leadership because it's not something I've been able to influence really.” (Trust 17, clinical lead, medicines management) |
2 Staff motivation and engagement | “they're [executives are] well equipped to give that person the idea of how to put it right themselves. Which really empowers them more and makes them feel an awful lot better, because then they realise that they can actually sort the problem out themselves, and they didn't have to go to somebody quite high up the Board to get it sorted. It was something that they could have done themselves.” (Trust 8, clinical lead, critical care) “we've got leadership rounds, and that's made a big difference to identifying the problems on the wards, but actually some of the problems have been given back to the wards when really we should be saying, this is common across the Trust, let's solve it by the Trust.” (Trust 13, clinical lead, medicines management) “We had such a problem with infection here, we were just desperate to do something about it and quite a lot of the, my more dapper colleagues, were very reluctant to shed their nice suits and shirts and, or to roll up the sleeves on their shirts because they didn't think it looked professional.. all the problems evaporated when the chief executive sent out an email inviting for a one-to-one interview any clinician who didn't wish to follow this particular policy, and I believe no one took her up on it.” (Trust 16, clinical lead, general wards) |
3 Commitment and support | “I certainly know that our Chief Executive has met with all the consultants in small groups..certainly [CEO] has said himself, if you've got problems then you come directly to me. If it's Safer Patient then you get straight access to me, and that has been really encouraging.” (Trust 1, clinical lead, general wards) “we would feedback the activities from the previous month, our anticipation of what would happen the following month and any issues that we were faced with, that we needed support from the leadership team. And whether that was a resource issue or something about can't get clinicians involved, whatever and that was fine” (Trust 14, director of nursing) |
4 Monitoring progress | “there's a quarterly report to the Trust Board.. the chief exec does a section as part of his report each month. And then [name] or I, or both, go and talk about something specific every quarter. So in December, it was the walk rounds and what we'd done there. And in, three months after that, whatever it was, March, February, March, we presented to them he Run Charts. And next time we'll do something different” (Trust 9, general manager) “[with CEO and management team] we will go through.. our traffic light measures.. which would show all of our measures then and then where we are with them. Green, we're passing the Run Chart rows, and the amber, where we aren't passing the rows just yet, and then the red is if we haven't got any data points against it.. what we do is pick on, put together a progress report, which is then brought to a Trust Board.. and generally during the meeting we can raise any concerns we may have about certain, about if there's any measures that we're struggling with” (Trust 10, programme coordinator) |
5 Embedding programme elements | “our new chief exec has made sure that safety is put on the agenda first, so she's also a very good driving force for it” (Trust 8, programme coordinator) “Go back, ask them to give you the Board agendas for about the last 18 months and you tell me where you see clinical governance. It was always down the pecking order.. it's now on the agenda, it's on the agenda as patient, as the SPI thing.. I've got the support of the chief exec” (Trust 11, medical director) |
CEO, Chief Executive Officer.
Despite the difference in weighting of the dimensions, the staff reports substantiated the activities reported by the CEOs, such as their work towards the application of the programme, attendance at learning sessions and leadership walk rounds (initially considered apprehensively by many frontline staff but later welcomed). Moreover, the staff reports offered further insight into why CEO involvement was important and what each dimension offered to them. For example, staff feedback and presentation to the CEOs on SPI data measures (in the form of high-level data and metrics in Run Charts and traffic light measures) and summaries of progress and future plans (through verbal presentations and written reports), were reported to provide awareness, recognition, solutions and direction from the CEOs. These were considered invaluable, especially the recognition of staff work, and staff conveyed their wish to avoid disappointing the CEO. This suggests benefits gained from subtle acts of listening to presentations, reading reports, understanding and acknowledging the difficulties faced in implementation. The CEOs may not have realised the power of such straightforward intangible acts.
While most staff agreed that their CEO was engaged in the process and that their described commitment was valuable, they also portrayed the role of the CEO as secondary and supplementary to their own role in SPI. The staff saw themselves as the true implementers of the programme, while the CEOs were perceived to be best placed to offer assistance in the form of organisation-wide messages (statements of importance of the programme), recognition, direction and trouble shooting. Staff expressed a preference for more involvement by their CEO on the dimensions outlined or more from this involvement. For example, remarks cited the disappointment at the lack of feedback and actions following the walk rounds. While examples supported CEOs claims that they empowered staff to fix problems themselves, staff also viewed this as CEOs disregarding the opportunity to make organisation-wide changes. Alongside this, some reluctance to ask for help was communicated by the staff. There was speculation that the CEOs were preoccupied with organisational restructures and foundation status or other higher priorities, that they had superficial reasons for being involved (ie, funding and profile) and that they were only concerned with a couple of aspects of the whole programme (meetings and walk rounds).
Lastly, the peer reports highlighted the following activities and benefits of the CEO involvement that were not emphasised by the CEOs themselves: ensuring the right people are nominated for the programme, acting as a figurehead when IHI visited and meeting with the CEO of their paired SPI organisation (the 20 organisations paired up to share learning), maintaining external links with primary care Trusts, and offering an organisational perspective across all four workstreams.
Discussion
All the CEOs in this study recognised the importance of their part in the SPI programme. The executives gave detailed accounts of their activities and perceived value they brought to all of the different stages of the process: from the initial application to start the initiative, through overseeing and encouraging the process, to its sustainability after resources diminished. This supports proposals that senior management make a significant contribution to quality and safety improvement initiatives in the healthcare setting.11–13 In exploring the parts played by the CEOs, five critical dimensions were identified: (1) resource provision; (2) staff motivation and engagement; (3) commitment and support; (4) monitoring progress and (5) embedding programme elements. Staff views of CEO involvement closely matched the dimensions that emerged from the self-reports by the CEOs; however, the dimensions of embedding for sustainability and resource provision did not surface as markedly and the weighting of the dimensions differed from the CEOs’ reports.
Managerial commitment was an expected finding considering the literature support for this inside and outside of healthcare.27 28 We identified manifestations of commitment from: attending SPI learning sessions; leadership walk rounds; prioritising safety on the Board agenda; talks explaining the programme; stamps of approval for programme practices and stating its purpose. On the latter, research has implied the relevance of senior managerial influences in building the right culture for improvement.15 While a few of the interviewees recognised their responsibility in this, neither they nor the staff define these activities. Recent articles offer managerial actions on producing a good patient safety culture,29 but less is known on creating the right culture for QI.
Studying the components of the senior management role in a hospital setting in the US, Bradley et al10 identified that the following manager-related variables affected their QI initiative: senior management engagement; management's relationship with clinical staff; the promotion of an organisational culture of QI; support of QI with organisational structures and procurement of organisational resources for QI. Our findings are in accord with theirs, although interestingly our CEOs made more reference to their role as a monitor of the process. This included reviewing SPI measures and ensuring that programme targets were met. Due to a divergence of perceived monitoring benefits by CEOs and staff, further understanding of the beneficial ways of monitoring could assist managers in how to best carry out this task.
There is much recognition that QI initiatives require an open and mutual communication between management and clinical staff.30 31 Our interviewees emphasised that the benefits of shared dialogue with clinical staff was both to receive input on quality and safety and to engage staff. Indeed, senior managers have been identified as holding a facilitating responsibility,32–34 including research from another study on the first phase of the SPI programme showing importance of management involvement and commitment.19 The present study confirms the earlier conclusions and shows that this entails motivating and empowering staff by providing them with more autonomy, reinforcing SPI-compliant behaviours and attendance at the learning sessions to learn about improvement practices. Such learning is supported by studies that recommend managers to enhance their QI knowledge.13 CEOs involvement in resource provision is also supported by research proposals that senior managers’ activities for safety include granting resources for a comprehensive safety programme and permitting staff time for safety.35 Our findings show that the most common resources authorised by CEOs for the SPI programme were time allowed for SPI work and training, data collection and data analysis support personnel, information technology tools, and an SPI coordinator to oversee the project. However, these were mostly prescribed by IHI, and, while CEOS were happy with their distribution, they otherwise may have chosen different areas to resource.
Finally, a role reported by the CEOs as essential to achieving sustained learning and outcomes involved embedding SPI activity and procedures into existing organisational systems, strategies and processes. However, apart from references to changing Board agendas, staff made no mention of any of these strategies in relation to CEO involvement. This may be because this aspect of CEO involvement is mostly unseen by staff or that CEOs have either communicated their tasks differently or exaggerated their work on this. Recommendations based on these findings are to: modify Board agendas and prioritise safety; integrate programme targets into mission statements and strategic objectives; strengthen lines of accountability and introduce reporting mechanisms associated with programme outcomes; and incorporate programme approaches into other existing initiatives. Change of structures and systems by management has been shown to assist in the sustainability of QI programmes.10 In other analyses of the SPI programme, its integration within organisational structures and processes featured dominantly within strategies to sustain it.23 Such tasks arguably fit within the remit of senior management and further support the argument that their activity is relevant to collaborative methods being sustained, even if it may or may have not been in this case study.11
Limitations
It is important to highlight that this research has not been able to assess any association between the CEOs’ roles and successes/failures of the SPI programme. It instead describes the CEOs’ self-reported contribution to the programme. These self-reports may be subject to social desirability bias, especially as the interviewees were involved in the application process to secure implementation and additional programme funding. In a previous research survey of 635 of the SPI participators (including the CEOs), not only did senior management and frontline staff have many divergent views on the programme's strengths, weaknesses and impact, but also the senior managers held overall more positive views than the frontline.22 24 Equally, the fact that this sample volunteered for this high-profile initiative brings with it a self-selecting bias that is arguably likely to have led to an overestimation of the involvement that senior managers at this level would typically engage in within most improvement initiatives in their Trusts. However we have tried to lessen this limitation with supplementary analysis with staff views of those involved in SPI.
The SPI programme achievements remain unclear. In a large formal evaluation of hospitals involved in the SPI programme, while gains in quality and safety were found, the gains were no larger than in the control hospitals that were not involved in the programme.36 In particular, there may have been improvements in specific areas in some hospitals which were not detected by the broader evaluation. The evaluators themselves further noted that large-scale effects may take a longer time to surface.36 As the SPI as a programme did not demonstrate overall improvement or elucidate which organisations performed better than others, it is difficult to link CEO self-perceptions with formal outcomes, and the existing data does not show clear enough trends for this analysis. Lastly, the sample size is relatively small yet can be judged respectable when considering that the interviewees included all but one of the CEOs in charge of all of the NHS Trusts that participated within SPI across the UK and when considering the low number of CEOs in the wider UK population compared with other healthcare professionals. Nevertheless, a larger sample that is less homogeneous would have strengthened the study and its findings.
Conclusion
This study addressed the call for more research-informed practical guidance on the role of senior management in QI initiatives and specifically identified critical dimensions of CEO involvement within the SPI. The findings show that the CEOs provided key participation considered to significantly contribute towards the SPI programme. The reports reinforce conclusions in change management and the safety literature that have stressed the importance of CEO involvement, and further provide new evidence for specific critical dimensions of CEO involvement. Queries raised include the tangible benefits of executive involvement in changing structures and embedding for sustainability and the practical steps to creating the ‘right’ environment for QI. In providing a case-study illustration of the type of involvement that senior management engage in within an improvement collaborative, and at what stages certain actions took place, the study imparts guidance for other managers at this level opting into a similar intervention. The framework presented here could provide the basis for a quantitative assessment of CEO engagement in QI programmes, which might be linked to trends in process and outcome changes. Future work could also explore patterns of the types of CEO involvement across successful and unsuccessful sites.
Supplementary Material
Acknowledgments
We would like to thank all the CEOs who participated in this study for their time. We would also like to thank Dr Jonathan Benn, Susan Burnett, Anna Pinto and Sandra Iskander for their great contribution to data collection and preliminary analysis. We are grateful to our funders, the Health Foundation and the National Institute for Health Research.
Footnotes
Contributors: AP, CV and SD developed the aims of this study. Data collection was overseen by CV and carried out by AP along with other members of a research team acknowledged in this manuscript. AP carried out data analysis on all transcripts. The qualitative model was considered and checked for face validity and interpretation by CV and SD. The manuscript was prepared by AP then circulated among CV and SD for revision and final approval of the version published.
Funding: This work was supported by the Health Foundation and the Centre for Patient Safety and Service Quality, is supported by the National Institute for Health Research. Monetary provision from The Health Foundation and the National Institute for Health Research.
Competing interests: None.
Ethics approval: Ethical approval was obtained from the NHS National Research Ethics Service Leicestershire, Northamptonshire and Rutland Research Ethics Committee 2. Reference no. 07/H0402/69.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: More quotations per dimension of CEO involvement can be made available on request to Parand, Anam at a.parand@imperial.ac.uk
References
- 1.Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med 1989;320:53–6 [DOI] [PubMed] [Google Scholar]
- 2.Langley GJ, Nolan KM, Nolan TW, et al. The improvement guide: a practical approach to enhancing organizational performance. San Francisco: Jossey-Bass Publishers, 1996 [Google Scholar]
- 3.Carey RG. Improving healthcare with control charts: basic and advanced SPC methods and case studies. Milwaukee, Wisconsin: ASQ Quality Press, 2003 [Google Scholar]
- 4.Schouten LMT, Hulscher MEJL, Everdingen JJEv, et al. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 2008;336:1491–4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bray P, Cummings DM, Wolf M, et al. After the collaborative is over: what sustains quality improvement initiatives in primary care practices? Jt Comm J Qual Saf 2009;35:502–8 [DOI] [PubMed] [Google Scholar]
- 6.Øvretveit J, Staines A. Sustained improvement? Findings from an independent case study of the Jonkoping quality program. Qual Manag Health Care 2007;16:68–83 [DOI] [PubMed] [Google Scholar]
- 7.Øvretveit J. Does improving care coordination save money: a review of research. London: Report prepared for the Health Foundation, 2011 [Google Scholar]
- 8.Marshall M, Øvretveit J. Can we save money by improving quality? BMJ Qual Saf 2011;20:293–6 [DOI] [PubMed] [Google Scholar]
- 9.Øvretveit J.2009. Does improving quality save money? A review of evidence of which improvements to quality reduce costs to health service providers. Health Foundation Report. http://www.health.org.uk/media_manager/public/75/publications_pdfs/Does%20improving%20quality%20save%20money.pdf (accessed 21 December 2012)
- 10.Bradley EH, Holmboe ES, Mattera JA, et al. The roles of senior management in quality improvement efforts: what are the key components? J Healthc Manag 2003;48:15–28 [PubMed] [Google Scholar]
- 11.Øvretveit J, Bate P, Cleary P, et al. Quality collaboratives: lessons from research. Qual Saf Health Care 2002;11:345–51 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Parker VA, Wubbenhorst WH, Young GJ, et al. Implementing quality improvement in hospitals: the role of leadership and culture. Am J Med Qual 1999;14:64–9 [DOI] [PubMed] [Google Scholar]
- 13.Øvretveit J.2009. Leading improvement effectively: Review of research: Health Foundation Report. http://www.health.org.uk/public/cms/75/76/313/560/Leading%20imrpovement%20effectively.pdf?realName=YUaqJk.pdf (accessed 21 December 2012).
- 14.Locock L. Maps and journeys: redesign in the NHS Birmingham. Birmingham: The University of Birmingham, Health Services Management Centre, 2001 [Google Scholar]
- 15.Savitz LA, Kaluzny AD. Assessing the implementation of clinical process innovations: a cross-case comparison. J Healthc Manag 2000;45:366–79 [PubMed] [Google Scholar]
- 16.Institute for Healthcare Improvement The Breakthrough Series: IHI's Collaborative Model for Achieving Breakthrough Improvement. Diabetes Spectr 2004;17:97–101 [Google Scholar]
- 17.Health Foundation, The Safer Patients Initiative, UK: http://www.health.org.uk/areas-of-work/programmes/safer-patients-initiative/ (accessed 17 Jan 2012)
- 18.Burnett S, Benn J, Pinto A, et al. Organisational readiness: exploring the preconditions for success in organisation-wide patient safety improvement programmes. Qual Saf Health Care 2010;19:313–17 [DOI] [PubMed] [Google Scholar]
- 19.Parand A, Burnett S, Benn J, et al. Medical engagement in organisation-wide safety and quality improvement programmes: experience in the UK Safer Patients Initiative. Qual Saf Health Care 2010;19:1–5 [DOI] [PubMed] [Google Scholar]
- 20.Benn J, Burnett S, Parand A, et al. Perceptions of the impact of a large-scale collaborative improvement programme: experience in the UK Safer Patients Initiative. J Eval Clin Prac 2009;15:524–40 [DOI] [PubMed] [Google Scholar]
- 21.Burnett S, Parand A, Benn J, et al. Learning about leadership from Patient Safety WalkRoundsTM. Int J of Clin Leadersh 2010;16:185–92 [Google Scholar]
- 22.Benn J, Burnett S, Parand A, et al. Factors predicting change in hospital safety climate and capability in a multi-site patient safety collaborative: a longitudinal survey study. BMJ Qual Saf 2012;21:559–68 [DOI] [PubMed] [Google Scholar]
- 23.Parand A, Benn J, Burnett S, et al. Strategies for sustaining a quality improvement collaborative and its patient safety gains. Int J Qual Health Care 2012;24:380–90. [DOI] [PubMed] [Google Scholar]
- 24.Parand A, Burnett S, Benn J, et al. The disparity of frontline clinical staff and managers’ perceptions of a quality and patient safety initiative. J Eval Clin Pract 2010;17:1184–90 [DOI] [PubMed] [Google Scholar]
- 25.Glaser B, Stauss A. The discovery of grounded theory: strategies for qualitative research. New York: Aldine, 1967 [Google Scholar]
- 26.Flick U. An introduction to qualitative research. 4th edn London: Sage, 2009 [Google Scholar]
- 27.Mastal MF, Joshi M, Schulke K. Nursing leadership: championing quality and patient safety in the boardroom. Nurs Econ 2007;25:323–30 [PubMed] [Google Scholar]
- 28.Flin R. ‘Danger—Men at Work’: management influence on safety. Hum Factors Ergon Manuf 2003;13:261–8 [Google Scholar]
- 29.Reiman T, Pietikainen E, Oedewald P. Multilayered approach to patient safety culture. Qual Saf Health Care 2010;19:e20. [DOI] [PubMed] [Google Scholar]
- 30.Parker LE, Kirchner JE, Bonner LM, et al. Creating a quality-improvement dialogue: utilizing knowledge from frontline staff, managers, and experts to foster health care quality improvement. Qual Health Res 2009;19:229–42 [DOI] [PubMed] [Google Scholar]
- 31.Atun RA. Doctors and managers need to speak a common language. BMJ 2003;326:655. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Weiner BJ, Shortell SM, Alexander J. Promoting clinical involvement in hospital quality improvement efforts: the effects of top management, board, and physician leadership. Health Serv Res 1997;32:491–510 [PMC free article] [PubMed] [Google Scholar]
- 33.Wilkinson JE, Powell A, Davies H.2011. Are clinicians engaged in quality improvement? A review of the literature on healthcare professionals’ views on quality improvement initiative: Health Foundation Report. http://www.health.org.uk/public/cms/75/76/313/2440/Are%20clinicians%20engaged%20in%20quality%20improvement.pdf?realName=y8NfS4.pdf (accessed 21 December 2012)
- 34.Taitz JM, Lee TH, Sequist TD. A framework for engaging physicians in quality and safety. BMJ Qual Saf 2012;21:722–8 [DOI] [PubMed] [Google Scholar]
- 35.Flin R, Yule S. Leadership for safety: industrial experience. Qual Saf Health Care 2004;13:45–51 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Benning A, Dixon-Woods M, Nwulu U, et al. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ 2011;342:d199. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.