Requirements for a quality dashboard: Lessons from National Clinical Audits

Abstract

Healthcare organizations worldwide use quality dashboards to provide feedback to clinical teams and managers, in order to monitor care quality and stimulate quality improvement. However, there is limited evidence regarding the impact of quality dashboards and audit and feedback research focuses on feedback to individual clinicians, rather than to clinical and managerial teams. Consequently, we know little about what features a quality dashboard needs in order to provide benefit. We conducted 54 interviews across five healthcare organizations in the National Health Service in England, interviewing personnel at different levels of the organization, to understand how national (UK) clinical audit data are used for quality improvement and factors that support or constrain use of these data. The findings, organized around the themes of choosing performance indicators, assessing performance, identifying causes, communicating from ward to board, and data quality, have implications for the design of quality dashboards, which we have translated into a series of requirements.

Introduction

Dashboards are a type of health information technology (HIT) that use data visualization techniques to support clinicians and managers in viewing and exploring data on processes and outcomes of care¹. A distinction can be made between clinical dashboards and quality dashboards. Clinical dashboards provide feedback to individual clinicians on their performance compared to standards or targets, with the aim of informing decisions about, and thereby improving, patient care². For example, a clinical dashboard may seek to reduce inappropriate prescribing of antibiotics and therefore present data to clinicians regarding how their rate of antibiotic prescribing compares to that of their colleagues3,⁴. In contrast, quality dashboards show performance at the ward or organizational level to inform operational decision making and quality improvement efforts⁵. Ideally they will provide feedback that can be used at both ward and organizational levels^6,7. It is thought that the visualizations provided by quality dashboards can lead to the identification of previously unnoticed patterns in data⁸, informing quality improvement initiatives, and more efficient and effective decision making⁹.

Healthcare organizations worldwide are increasingly using quality dashboards as a way of monitoring the quality of care they provide and as the basis for quality improvement initiatives. For example, use of quality dashboards has been reported in the US7,¹⁰, Canada¹¹, the UK⁶, and the Netherlands¹². This is a trend that is likely to continue, with electronic health records providing a source of data to automatically populate quality dashboards¹³. Quality dashboards have been shown to have positive effects on performance indicators¹¹ and use of quality dashboards has been identified amongst boards of high-performing hospitals¹⁰. However, empirical evidence regarding the impact of quality dashboards remains limited. In a systematic review of the evidence base for the use of quality and clinical dashboards, 11 research studies that had evaluated their impact on either quality or clinical outcomes were identified¹ but only one of these studies met the definition of a quality dashboard¹⁴. Consequently, we know little about what features a quality dashboard needs in order to stimulate quality improvement.

Research on audit and feedback provides a number of suggestions that are likely to have relevance when considering the design of a quality dashboard. For example, Contextual Feedback Intervention Theory (CFIT) suggests that the recipient of the feedback needs to perceive the standard or target as desirable (goal attractiveness) and achievable (goal expectancy) and perceive the feedback about the discrepancy between their performance and the standard as accurate¹⁵. CFIT contends that such feedback is more likely to change behavior if it is timely, frequent, cognitively simple, e.g. presented graphically, unambiguous, and provides concrete suggestions of how to improve performance. Hysong et al.’s model of actionable feedback suggests clinicians are more likely to respond to the feedback if it is perceived to be timely and non-punitive and if they receive feedback about their own individual performance rather than aggregated data about overall performance of the organization¹⁶. If they are able to customize how they view the data, this leads to active engagement in sensemaking, further increasing the likelihood that the feedback will be acted on. However, as the audit and feedback literature is largely concerned with feedback to individual clinicians¹⁷, this raises the question of how such recommendations can be adapted for quality dashboards, given that quality dashboards provide feedback to teams, whether clinical teams or managerial teams.

In the UK, the Healthcare Quality Improvement Partnership (HQIP) centrally develops and manages a programme of over 60 national clinical audits each year through the National Clinical Audit and Patient Outcomes Programme (NCAPOP). The National Health Service (NHS) standard contract requires all healthcare organizations that provide NHS services to participate in the NCAPOP national clinical audits and they are required to contribute to their funding. In addition, there are over 50 independent national clinical audits that are not part of the NCAPOP, funded either through subscription, by a charity or professional body, or by NHS England (the body responsible for commissioning NHS services in England). In comparison to Hospital Compare¹⁸ in the US, which provides an overall hospital rating made up of 57 measures, in the UK each national clinical audit focuses on a particular clinical area or condition, making them more comparable to national registries. Each national clinical audit produces an annual report that is made available to the public. National clinical audits aim to systematically measure the quality of care delivered by clinical teams and healthcare organizations and to stimulate quality improvement¹⁹.

While there is evidence of positive impacts of national clinical audits^20,22, there is variation both within and between healthcare organizations in the extent to which they engage with national clinical audit data^23,24. Consequently, national clinical audit data are substantially underutilized and the potential for national clinical audit data to inform quality improvement is not being realized. In response to this, we are undertaking a study to develop a quality dashboard for exploring national clinical audit data by both clinical teams and managers. As part of this, we undertook an interview study with clinicians, managers, and audit support staff to understand how national clinical audit data are currently used for quality improvement and factors that support or constrain the use of these data.

In this paper, we draw on the interview data to identify themes that have relevance for the design of quality dashboards. We first describe the methods of the interview study and then present our results. We conclude by discussing the implications of our findings for the design of quality dashboards which could be used across healthcare contexts, translating them into a series of requirements for future dashboard design.

Methods

Participants

Interviews were conducted across five acute healthcare organizations in England. Three were teaching hospitals and two were district general hospitals. To ensure our findings were generalizable beyond a single national clinical audit, interviews explored use of the Myocardial Ischaemia National Audit Project (MINAP) and the Paediatric Intensive Care Audit Network (PICANet), which vary in terms of clinical specialty, patient groups, and performance indicators. MINAP has been running continuously since 2000 and data are contributed by all hospitals in England, Wales and Northern Ireland that admit patients with acute coronary syndromes²⁵. PICANet was set up in 2002 and contains data from all NHS Pediatric Intensive Care Units (PICUs) in England and Wales²⁶. Examples of the performance indicators included in these audits are provided in Table 1. As the two district general hospitals did not have PICUs, use of PICANet data was investigated in PICUs across three of the healthcare organizations and use of MINAP data was investigated within cardiology services across all five organizations. As both MINAP and PICANet are part of the NCAPOP, to capture further variation, we also gathered information on use of the British Association of Urological Surgeons (BAUS) audits and the National Audit of Cardiac Rehabilitation (NACR), which are independent audits.

Table 1.

Example performance indicators for MINAP and PICANet.

	MINAP	PICANet
Performance measures	•Call (by patient/relative to emergency services) to balloon (percutaneous coronary intervention) time;•Door (arrival in Emergency Department) to balloon time; time to angiography;•Medication on discharge;•Patient seen by a cardiologist	• Accidental extubation;• Length of stay;•Admissions (planned or unplanned);• Refusal of admission
Outcome measures	•30 day mortality for ST-elevation myocardial infarction patients•Complications such as bleeding, stroke	•Risk adjusted standardized mortality ratio;• Emergency readmission within 48 hours

A combination of purposive and snowball sampling was used. In each site the clinical contact for the study (typically a MINAP or PICANet lead) was interviewed first. These contacts were then asked to identify others who were involved with national clinical audits, enabling us to map the networks of stakeholders through which data for particular audits were captured, accessed, and analyzed. We sought to interview participants at all levels of the organization, including clinical team members, Quality & Safety Committee members, and members of the Board of Directors. We also sought to interview those who commission healthcare services from the five sites.

Ethics approval for this study was received from the University of Leeds, School of Healthcare Research Ethics Committee (Approval no. HREC16-044).

Data collection

A semi-structured interview topic guide was developed by the research team. This was reviewed by the study Lay Advisory Group and revised based on their feedback to ensure that the interviews explored topics that matter to patients. Interviews began with discussion of the participant’s role, with a focus on their responsibilities for national clinical audits (e.g. collection, analysis, dissemination). Participants were then encouraged to draw on their own experiences and articulate how national clinical audit data were used, with an emphasis on the role of the data in informing quality improvement. They were asked about the circumstances that they felt constrained or supported use of national clinical audit data for quality improvement and what features they would consider important in a quality dashboard for exploring national clinical audit data. Example interview questions are provided in Table 2. Interviews were audio recorded and transcribed verbatim.

Table 2.

Example interview questions.

Which [MINAP/PICANet] data do you think are important for making assessments of care quality and informing quality improvement initiatives?

Probes: How do these measures help you to make an assessment of care quality? What are you comparing against (national average, reference standard)? Probe for differences in the perception of value/actionability of process versus outcomes data

How are the [MINAP/PICANet] data used?

Probes: Who accesses the data? In what format are the data accessed? What is focused on and why? Is there a process by which audit data are regularly reviewed? Do you have processes for disseminating the information to others?

Have the [MINAP/PICANet] data been used to make any changes in practice/inform quality improvement initiatives? (Explore through examples if possible)Probes: How are decisions made on what changes need to be made/who was involved? Why were the changes instigated? How were the changes introduced? Were they made in a timely manner? Do you think the changes were successful?

Are there any obstacles/challenges to using [MINAP/PICANet] data?

Probes: How data are accessed? How data are presented? Are senior management/staff supportive and engaged in the use of audit data? If data quality, what gets in the way of data quality? If lack of timely data, how would more timely data improve things?

Do you think a quality dashboard for exploring [MINAP/PICANet] data would be useful?

Probes: If not, why not? What data will it need to present to be useful? How should data be presented? What features will the dashboard need to have to be useful?

Data analysis

The interviews were anonymized and entered into NVivo 11. Framework analysis, an approach developed for analyzing qualitative data for applied policy research, was used²⁷. Informed by the interview topic guide and reading of preliminary interviews, codes for indexing the data were identified and agreed by four members of the research team. They then indexed 5 transcripts to test the applicability of the codes and assess agreement. Where there was variation in the indexing, the codes were refined and definitions were clarified. The refined codes were applied to all transcripts. Example codes are provided in Table 3. The indexed data were summarized in a matrix display to build up a picture of the data as a whole²⁸. In the final stage, mapping and interpretation, the matrix was used to identify similarities and differences in participants’ responses.

Table 3.

Example codes for indexing data.

Supports or constraints on engaging with audit data
	a.	Presentation of data
	b.	Timeliness of data
	c.	Data quality
	d.	Access to data
	e.	Dissemination and monitoring processes
	f.	Resources
Quality dashboard
	a.	Functional tasks
	b.	Data requirements
	c.	Mandated constraints
	d.	Look and feel requirements
	e.	Usability requirements
	f.	Performance requirements
	g.	Security requirements
	h.	Implementation requirements
	i.	Potential impact of a quality dashboard

Results

Fifty-four interviews were conducted between November 2017 and June 2018. Interviews ranged from 33 minutes to 1 hour 29 minutes, with an average (mean) length of interview of 54 minutes. Table 4 provides a summary of participants by professional group and audit. A significant number of participants, such as members of Quality & Safety Committees and Board members, used multiple audits.

Table 4.

Participants by professional group and audit.

		MINAP	PICANet	BAUS	NACR	Multiple audits	Total
Professional group
	Physician	7	5	1			13
	Nurse	6	3		2	4	15
	Audit support staff	1	3				4
	Board members					2	2
	Quality & Safety staff					8	8
	Information staff					6	6
	Commissioners					4	4
	Other		1			1	2
Total		14	12	1	2	25	54

Analysis of the interviews revealed five main themes that have significance for the requirements for a quality dashboard: choosing performance indicators, assessing performance, identifying causes, communicating from ward to board, and data quality. The requirements for each theme are summarized in Table 5.

Table 5.

Requirements for a quality dashboard.

Theme	Requirements
Choosing performance indicators	1. Allow users to select which performance indicators are displayed
Assessing performance	2. Where evidence-based standards exist, make it easy to assess how performance compares to that standard
	3. Support identification and evaluation of trends over time
	4. Allow users to select the time period over which performance indicators are displayed
	5. Support comparison against the national average
	6. Allow users to select particular organizations to compare with
Identifying causes	7. Enable users to ‘drill down’, e.g. to look at particular subgroups of patients
	8. Provide access to information about other clinical areas within the organization
	9. Support simultaneous interaction for discussion at the clinical team level
Communicating from ward to Board	10. Enable easy identification of when a clinical area is an outlier within a particular audit
Data quality	11. Provide timely data
	12. Use sources of data that staff trust

Choosing performance indicators

A key issue in the design of any dashboard is the choice of performance indicators to be displayed. In discussing this with our interviewees, they talked about wanting a dashboard that was customizable, in terms of enabling them to select what performance indicators are displayed (Requirement 1). Such an approach would enable users to select performance indicators that they perceive as important signifiers of care quality. In looking at use of national clinical audit data, we found that there was limited engagement from nursing staff in part because the national clinical audits did not capture what they considered to be important markers of care quality. For example, in the PICUs, nurses were concerned about the number of central line infections, data that are not captured in PICANet. When deciding which performance indicators should be displayed on a quality dashboard, it is important that an interdisciplinary decision is made and that the selected performance indicators have relevance to all members of the multidisciplinary team.

Interviewees also talked about wanting to have access to performance indicators from other areas of the organization, a topic we return to under ‘Identifying causes’.

Assessing performance

Quality improvement depends on using the selected indicators to make comparisons, thereby providing an assessment of performance, as this interviewee describes:

Quality improvement is about having a process in place where you are measuring those things, whether it’s patient surveys or hard data-type audit, and comparing it against either a set national standard or your peers or benchmarking […] and that you then have a system in place where that comparison allows you to reflect on how you are performing and make the necessary adjustments to improve performance. (Site 2, Cardiologist 4)

We identified three key ways in which clinical teams assessed their performance, which have implications for the information that should be provided in a quality dashboard and how data should be presented. One way in which performance was assessed was comparing practice against evidence-based standards that are known to improve patient outcomes (Requirement 2):

I do reports, like monthly and yearly, to show how we’re doing. […] it’s the nurses’ data, but I do all the data to tell them if they’re on schedule to hit the 60% target [angiogram within 72 hours for non ST-elevation myocardial infarction (NSTEMI) patients] and stuff like that. I look at all the figures for the STEMIs [ST-elevation myocardial infarction], to see if they’re hitting the 150 minute call-to-balloon and 90 minute door-to-balloon. (Site 4, MINAP Audit Assistant)

However, such an approach only works in clinical areas where there are evidence-based standards against which to assess performance. Additionally, research on audit and feedback interventions aimed at individual clinicians emphasizes that clinicians need to see meeting these standards as achievable¹⁵.

The second way in which performance is assessed is through monitoring trends over time (Requirement 3). This type of monitoring is particularly important for clinical areas where there are no evidence-based standards against which to compare service performance, as is the case in pediatric intensive care. Consequently, we found that clinical teams would use PICANet data to monitor trends over time (month-on-month):

We use the PICANet data to produce monthly graphs that we use at our monthly clinical governance meetings. So basically, we look at re-admissions within 48 hours of discharge, deaths, accidental extubations. So, we get graphs with month-on-month numbers, so we can monitor it and see changes. (Site 1, Pediatrician 2)

The interviewee described how monitoring PICANet data in this way led to a quality improvement initiative to reduce the number of accidental extubations; having identified a ‘spike’ in accidental extubations, additional training was introduced for nursing staff, about how to tape the tube to keep it in position and how to check the tube position on a chest x-ray.

While such assessments of performance over time were only undertaken in those sites where they had the resources to generate appropriate graphs, particularly in terms of staff skills and time, a quality dashboard should allow easy identification of such trends. Interviewees also talked about wanting to be able to choose over what time period a particular metric was displayed (Requirement 4); for example, for metrics where you may expect to see fluctuations over a year, it would be necessary to view data for several years.

The third way in which performance was assessed was through comparison with other healthcare organizations (Requirement 5). National comparisons offer opportunities to identify high-quality services in other organizations, which could act as examples of good practice from which clinical teams can learn. Making such comparisons was important to all sites, and perhaps especially to PICUs, which do not have access to evidence-based standards and thus feel the need to benchmark themselves against other units to assess their performance.

Clinical teams, in addition to comparing themselves against the national average, may also want to be able to select particular organizations to compare themselves against, organizations that they consider similar in terms of size and/or case mix (Requirement 6). One interviewee made the following comment:

There’s PICUs that, you know, if they had no re-admission rate, I wouldn’t be surprised because it’s not a high level PICU. So, you’d have to be stratified to a PICU, an average of a PICU who is like you. So, we have cardiac patients, we have neuro patients, we have ECMO patients, so we can only really be compared with them, otherwise we’re not comparing like with like. (Site 1, Pediatrician 2)

Identifying causes

Having assessed performance, a quality dashboard should then support clinical teams to identify causes for any particular performance indicators that raise concern. Interviewees described wanting to be able to ‘drill down’ into the data to understand the reasons behind the numbers, for example, to look at a particular sub-group of patients (Requirement 7). Drilling down may also go as far as looking at details of individual patients:

It’s looking at the complications we’ve had and how we can improve them. So it’s things like the patient’s died: was it due to the procedure or was it something else? And we look at all the pathway, you know, should they have come to us, should they have come quicker? You know, we then tweak things to improve it. Or it might be that they’ve had a dissection or a blockage while they’ve been doing the procedure, and we look at, well, what equipment was used? And again, we feed back to each individual operator on their cases. So it might be Doctor so-and-so: ‘we think you should have used this piece of kit when you got into that difficulty, we’ve tried…you know, another colleague’s tried this and it worked’. So then they’ll know next time to think it. (Site 1, MINAP Audit Support Nurse)

Interviewees at the clinical team level also talked about wanting to see data from other clinical areas, rather than data in silos, to understand how their performance interacted with and was impacted on by other areas of the organization (Requirement 8). For example, if a PICU receives a large number of emergency readmissions from the High Dependency Unit (HDU), they may also want to look at performance indicators for the HDU.

Some interviewees at the clinical team level talked about wanting a dashboard that could support simultaneous interaction by multiple users, so as to be able to drill down into the data and discuss within a meeting (Requirement 9). The interviewees felt that, through this, a quality dashboard could enhance the ability of teams to engage flexibly and in depth with national clinical audit data:

The potential is you could use it in further discussions or using that information to probably motivate you or stimulate you into something in an audit group, so it might be that there’s an aspect of care which […] another unit might be doing really well so you might want to say: well, how are they doing that? So having that access, that would probably be something which we’d do on a larger group, that’s what I would think. (Site 1, PICU nurse)

This may mean making the dashboard available on large touchscreens or tabletop computers, or enabling multiple users to access it via laptops and handheld devices.

Communicating from ward to board

For monitoring of care quality, it is important that data such as those contained within national clinical audits move from ‘ward to Board’⁶. However, the requirements at these different levels of the organization vary significantly. At divisional and corporate (Board and sub-committees that report to the Board, such as Quality and Safety Committees) levels, a different set of performance indicators will be of concern, those which reflect the performance of the division or organization as a whole. For example, interviewees at these levels discussed looking at performance indicators related to mortality, never events, harms, and complaints. There may be a disconnect between the performance indicators considered at divisional and corporate levels, which are likely to reflect national priorities, and those that are considered at the clinical team to reflect care quality.

Nonetheless, our interviewees reported that there are occasions when the national clinical audits are considered at the corporate level, particularly where a clinical area is identified as an outlier; as one Board member at Site 1 said ‘Nobody wants to be an outlier.’ One way in which a quality dashboard could support the use of such data at corporate levels is by enabling easy identification of when a clinical area is an outlier within a particular audit (Requirement 10):

What I need is flags that tell me and direct me to things that are outliers that I need to look at and then I can interrogate them […] all of us are so pressured that we absolutely need things that are going to say: ‘problem here, here’s how you look for it’, and then you go in and look for it. (Site 2, Quality & Safety Committee member)

One suggestion for achieving this was the use of the ‘traffic light’ or ‘RAG-rating’ system, where red, amber or green are used to indicate performance that falls below, almost falls below, or meets required standards. This was favored by several interviewees, because it is used frequently within healthcare organizations and enables users to identify potential problems at a glance. However, several interviewees also pointed out limitations with such an approach, most notably that it does not capture nuances within data, as when a standard has not been achieved for sound clinical reasons.

While it has been suggested that high performing hospitals are those where Boards spend time discussing quality issues²⁹, we found that, at divisional and corporate levels, full agendas meant there were significant constraints on time available in meetings for discussion. Where clinical team level performance indicators are considered in meetings, they will not be considered in detail. Consequently, the level of interaction and the ability to drill down that is required at the clinical team level is not needed within divisional and corporate level meetings (although will be required outside of meetings).

Data quality

Data quality was a topic that was frequently discussed in the interviews, with key issues being timeliness and trust in the data. At all levels, a key constraint in use of national clinical audit data for quality improvement was lack of access to timely data, with the data contained within annual reports often being perceived as too out of date to be of use. Timely data, that reflects current performance, were considered essential if they are to inform quality improvement (Requirement 11). While organizations may have timely data about their own performance, timely comparator data are also needed.

For clinical teams to engage with data for quality improvement, it is important that they have trust in the quality of the data (Requirement 12). For example, in one of the sites, they employed a dedicated PICANet audit co-ordinator who entered data into the Access database with only a day’s delay and, with support from the clinical lead, checked the accuracy and completeness of the data. Given these resources and supports, the pediatricians had confidence in the accuracy of data reports generated from the local database and used them to inform clinical governance meetings.

Trust in the quality of national comparator data is also important for clinical teams. Inconsistent or inaccurate coding was reported to have a negative impact on interviewees’ trust in MINAP data and on their ability to make meaningful comparisons with other organizations. Indeed, owing to inconsistent coding, one interviewee likened such assessments to ‘comparing apples and oranges’ (Site 3, Cardiologist).

Discussion

Quality dashboards are increasingly being introduced into healthcare organizations as a tool to support quality improvement, yet limited empirical research on use of quality dashboards means that we know little about what features a quality dashboard should have in order to provide the desired benefits. We have drawn on the experience of a range of professional groups who have experience of working with national clinical audit data in the NHS in England to understand what supports and constrains the use of such data for quality improvement. The findings suggest a number of requirements for a quality dashboard, which we have summarized in Table 5.

A number of the findings echo findings in the audit and feedback literature concerning the characteristics of feedback that are most likely to trigger a change in behavior. For example, the need for feedback to be timely is described in CFIT¹⁵ and the model of actionable feedback¹⁶, and is also highlighted more generally in the literature³⁰. Similarly, credibility of the information is also noted as an important characteristic. The desire of interviewees to be able to select the performance indicators displayed can be seen as contributing to the ‘goal attractiveness’ that CFIT describes as influencing the likelihood of action¹⁵ because it enables users to select performance indicators that they perceive as important signifiers of care quality. It also enables them to select performance indicators that are consistent with established goals and priorities, another feedback characteristic that has been highlighted in the literature³⁰. Similarly, comparing performance against evidence-based standards, those that are known to lead to improved patient outcomes, may also contribute to enhancing goal attractiveness.

However, the findings also highlight some differences that have relevance when designing HIT to provide feedback to a team, rather than to an individual. For example, selection of performance indicators needs to be agreed through an interdisciplinary approach and it is necessary to ensure that they have relevance to all members of the multidisciplinary team. To support in depth engagement and discussion as a team, support for simultaneous interaction by multiple users is likely to be required and there are also implications for the hardware on which the dashboard is displayed. Literature on audit and feedback suggests that feedback should be constructed through social interaction³⁰, which is hard when it is an individual clinician interacting with a clinical dashboard but which can and should be facilitated by a quality dashboard. We also see differences between the requirements when providing feedback to an individual clinician about their performance and when providing information to managers about the performance of a clinical area. Here what becomes important is easy identification of when a clinical area is not meeting particular targets or standards.

We have described a number of ways in which clinical teams assessed performance: comparing performance to a standard, assessing change over time, and comparing performance to performance of other healthcare organizations. While which of these is most effective remains to be determined³⁰, we did identify that the choice of approach and choice of organizations against which to compare will depend on the clinical area. Specifically, comparison against a standard is only likely to be effective if there is evidence to demonstrate that the standard is associated with improved patient outcomes. Existing literature on audit and feedback suggests that only one method of comparison should be used, to avoid the risk of conflicting messages, e.g. if performance is shown to be improving over time but does not meet the desired standard. However, we would argue that both comparing performance to a standard and assessing change over time are important because, depending on the organization’s starting point, the standard may take years to achieve yet it is appropriate for clinical teams to see that progress is being made.

Limitations

For healthcare organizations that are looking to develop quality dashboards, a prioritized list of requirements would be helpful, particularly if resources might be limited. While we have described how the relevance of the requirements will vary according to the user and the clinical area, the qualitative approach to data collection that we have undertaken does not allow us to provide a prioritized list. In a workshop with representatives of 22 national clinical audits, Requirements 2, 4, 7 were categorized as essential by all participants and scoring of requirements identified Requirements 1, 4, 7, and 11 as top priorities. However, the perspectives of the national clinical audits who participated in the workshop may differ from the perspectives of clinical teams and staff at divisional and corporate levels, and thus it would be beneficial to repeat this exercise with them.

A limitation of this research is that data were collected through interviews. Thus, while we have identified requirements for a quality dashboard that our interviewees perceived as important in order to support quality improvement, we do not have empirical evidence to demonstrate that a quality dashboard that meets these requirements will lead to quality improvement. In later phases of this study, we will be introducing a quality dashboard into the five organizations and evaluating its impact, allowing us to assess the importance of the requirements identified.

This research was conducted in the UK, which has an established mechanism for collecting data nationally for quality improvement purposes. However, while it might be UK focused, the findings regarding the implications for developing quality dashboards are relevant across healthcare organizations and contexts.

Conclusion

This study has explored how national clinical audit data are currently used for quality improvement and factors that support or constrain the use of these data. The findings, organized around the themes of choosing performance indicators, assessing performance, identifying causes, communicating from ward to board, and data quality, provide a number of requirements to inform the design of quality dashboards. Future research will explore the extent to which the described features support engagement with national clinical audit data and stimulates quality improvement.

Acknowledgements

This research is funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) Programme (project number 16/04/06). The views and opinions expressed are those of the authors and do not necessarily reflect those of the HS&DR programme, NIHR, NHS or the Department of Health.