Variations in Clinical Audit Collection: A Survey of Plastic Surgery Units Across the British Isles

Abstract

INTRODUCTION

Clinical audit is a requirement of good medical and surgical practice and is central to the UK Government's plans to modernise the NHS.

MATERIALS AND METHODS

A survey was conducted to assess clinical audit data collection and collation within plastic surgery departments across the UK. The survey identified a variety of different data collection and collation methods, with extensive differences between plastic surgery departments. Those responsible for data collection and its funding were also identified by the survey.

RESULTS

Results were obtained from 45 plastic surgery departments. Of the 45 departments surveyed, 12 collect data prospectively, whereas 26 units collect data retrospectively. The remaining departments collect data using a combination of methods. Of the units surveyed, 28 collect data on paper-based systems, with only 13 units using electronic applications. The personnel responsible for data collection were identified as being junior doctors. Departments collecting data prospectively do so from a greater number of sources than those collecting data retrospectively.

CONCLUSIONS

This survey has focused on plastic surgery. The authors believe that similar results would be obtained from a survey of other surgical specialties. A huge variation in all parameters relating to the collection and collation of clinical audit data is seen. There are few standards within this specialty for data collection. Much work must be done in order to reach targets set by the UK Government.

The collection of patient data in a format that will provide an improvement in accuracy, availability and completeness of patient events is integral to the Department of Health's plans for a modern NHS.¹ The importance of these issues was outlined by the Department of Health (DH) in a White Paper entitled Information for Health. This paper, published in 1998, describes the transition between the information systems in operation today and those expected to be available across the country by 2006.¹

It is essential for all clinicians to be involved in clinical audit in order to improve patient care and outcomes.² Recent events mean that it is a necessity that figures are kept and analysed within NHS trusts. This will keep standards to the highest level possible and ensure the early identification of those areas experiencing difficulties. It is as important for surgeons, as it is for physicians, to collect data regarding their activity and outcomes, although the manner in which data is collected and collated may differ significantly.³

Materials and Methods

A national telephone survey was used to survey all plastic surgery units in the British Isles. This survey was structured around direct questions relating to clinical audit. The manner, method and source of data collection were documented, along with present funding, source of data and the personnel responsible for data collection.

Results

Participants

Forty-five of the 47 departments surveyed responded to our questions.

Manner of data collection

The manner in which clinical data is collected within departments was investigated. Twelve of the 45 (27%) departments collect data in a prospective fashion, that is to say that data is collected and stored on a daily basis. Twenty-five of the 45 (56%) departments collect data in a retrospective fashion and eight departments (17%) collect patient data using a mixture of the methods (where different audit data is not collected in a standard fashion) as summarised in Table 1.

Table 1.

Differences in data collection

Manner of data collection	Number of units
	Electronic	Paper	Combination	Total
Prospective	8	4	0	12
Retrospective	3	20	3	26
Combination	2	4	1	7
Total	13	28	4	45

Method of data collection

The method by which data is collected and then stored for future evaluation was analysed, whether it was in electronic form, paper form or a combination of paper and electronic methods. Thirteen (28%) units utilise an electronic method, whereas 28 (62%) units use a paper method for data collection (Table 1).

The manner in which electronic data can be stored and used varies greatly. Of the 17 (38%) plastic surgery departments using electronic systems, 5 (29%) use a database, 10 (59%) use spreadsheets and 2 (12%) use a word processing based system (Fig. 1).

Figure 1.

Different types of electronic data collection.

Difference in groups

A difference was observed between the groups collecting data retrospectively and those collecting data prospectively. The majority of the retrospective group use paper as their storage and collecting medium compared with the majority of the prospective group who use electronic forms of data collection (Table 1).

Sources of data

There is also a difference between these two groups when considering the number of sources from which they both gather data for interpretation. Five different areas that generate clinical data were identified and the number of sources used by individual units noted. Across all units, the survey reports that 41 of the 45 (91%) departments collect data on the ward, 22 (49%) in out-patients, 21 (47%) in dressing clinics, 9 (20%) in accident and emergency and nine (20%) in theatres. The retrospective group was found to use less sources compared to the prospective group (Fig. 2).

Figure 2.

Difference in number of data sources.

Use of coded information

Fifteen (33%) units use a selection of coded data for the purposes of departmental audit. Eleven (24%) use codes for activity data, 10 (22%) use coded complication data, 14 (31%) use codes in the generation of log books and 5 (11%) use coded data for the purposes of generating discharge summaries.

Responsibility of data collection

The persons responsible for the collection of audit data are primarily the unit juniors. In 31 departments (69%), the senior house officers are responsible and registrars are responsible in 27 units (60%). In almost half of the units, a combination of people collect data. Nurses, senior doctors and data clerks appear to be used less in the collection of audit figures.

Discussion

Increasing information technology (IT) use within the clinical arena is one of the DH objectives for a new NHS. Such use of IT may improve patient outcomes by a variety of means. Comprehensive patient records held in electronic formats (electronic patient records, EPR) will allow physicians and surgeons to share prmation instantaneously over great distances.

EPR accuracy is dependant on data input. The importance of a clinician-based approach is well documented in the development and implementation of information systems.^2,3 It is hoped, ultimately, that such systems will provide data relevant to efficiency, outcomes, service performance and for continuous professional development.⁴ Applications such as these will also enable healthcare professionals to access real-time medical histories at the point of patient care.

Current methods of clinical audit data collection within plastic surgery departments vary tremendously. Methods by which data are collected range from paper systems through to electronic databases. The majority of units use paper systems and collect data retrospectively. There is no standard format for electronic data collection. Electronic applications range from basic word-processing documents through to databases, which provide users with powerful data searching and analysis.

Interesting differences have emerged between a group of units who collect data retrospectively and a group who collect it prospectively. Prospective data collecting units are more likely to use electronic systems and collect data from a greater number of sources.

Diagnostic and therapeutic codes are unpopular. Fifteen departments use them in some form or another. Two of the most important parameters in measuring outcomes, coded activity and complication data, are used in 24% and 22% of departments, respectively.

Although this survey focuses on plastic surgery, the authors believe that similar results would be found in other surgical specialties. Presently, there are no standards available which could be adopted by different specialties to allow more consistent data collection and interpretation. National datasets will help; however, their focus is limited and they are not designed to impact on the methods chosen to collect clinical data.

A standardised approach

Criticisms, cited in the Department of Health's White Paper of 1998 regarding information technology within the NHS, included wasting data and the collection of irrelevant information. However, there has been no apparent move over the past 7 years towards standardising our approach to clinical audit in order to reduce the waste and ensure that only relevant data are recorded.

Even within a relatively small specialty, there are no minimum standards in the methodology of clinical data collection for the purposes of audit. It is imperative, if we are to meet UK Government standards, that we adopt a standardised approach to data collection and collation. The data collected and produced must reflect performance reliably so as to withstand public scrutiny and our own analysis. Standardised data collection techniques do not exclude variations in local data collection where appropriate, and can still be flexible. Such datasets, however, should mean that comparisons between units could be meaningful and constructive.

Junior doctors in the survey are the group most likely to take on the responsibility of data collection. However, previous work has shown that junior doctors introduce greater inaccuracy in audit data.⁵ It is essential, therefore, that junior staff be educated in the audit process, to improve the reliability and accuracy of data.⁶ The workload associated with producing quality data should not be underestimated and, with a reduction in the current working hours, the proportion of time spent on audit will increase. No one can yet estimate what cost this will have on the training of the workforce, and no effort has been made thus far to try and facilitate clinical data collection to address this issue.

The utilisation of diagnostic and therapeutic coding systems in clinical audit is contentious. Coding sets have been criticised by medical staff for being cumbersome or unworkable. They remain, however, central to the role of facilitating data analysis and are part of the UK Government's objectives.^1,6

Conclusions

The authors believe that to meet UK Government objectives, changes are required to the way clinical audit is performed. Agreement on data sets needs to be sought. The workforce should be educated on the importance of clinical audit and the methods by which it can be conducted. Funding is needed to develop clinical audit. Coding standards need to be introduced to assist data analysis and standardised methods need to be sought to facilitate data collection and collation.

This is a tough brief, especially when considering the time scales involved. To ensure audit systems provide quality data, audit should be conducted in a standardised manner, not in the fragmented unstructured way that is the current practice.