Clinical audit, a valuable tool to improve quality of care: General methodology and applications in nephrology

Abstract

Evaluation and improvement of quality of care provided to the patients are of crucial importance in the daily clinical practice and in the health policy planning and financing. Different tools have been developed, including incident analysis, health technology assessment and clinical audit. The clinical audit consist of measuring a clinical outcome or a process, against well-defined standards set on the principles of evidence-based medicine in order to identify the changes needed to improve the quality of care. In particular, patients suffering from chronic renal diseases, present many problems that have been set as topics for clinical audit projects, such as hypertension, anaemia and mineral metabolism management. Although the results of these studies have been encouraging, demonstrating the effectiveness of audit, overall the present evidence is not clearly in favour of clinical audit. These findings call attention to the need to further studies to validate this methodology in different operating scenarios. This review examines the principle of clinical audit, focusing on experiences performed in nephrology settings.

Core tip: Clinical audit is a part of the continuous quality improvement process. It consists in measuring a clinical outcome or a process against well-defined standards, established using the principles of evidence-based medicine. The comparison between clinical practice and standards leads to the formulation of strategies, in order to improve daily care quality. This review examines the basis of clinical audit and the data about the efficacy of this methodology, focusing on nephrology issues. We think that clinical audit could offer to the modern Nephrologists a useful tool to monitor and advance their clinical practice.

INTRODUCTION

“Audit” is a Latin word, and the verb audio (‘hear’) indicates both active listening and the action of investigation and interrogation of the judiciary. Transferred to the English vocabulary “audit” takes on a meaning of “an official inspection of an organization’s accounts, typically by an independent body”[1].

The term is nowadays widely used in different settings (economic, business, etc.) referring to procedures aiming to ensure that the activities carried out for a purpose are consistent and effective for the achievement of objectives. Clinical (or medical) audits are part of the continuous quality improvement process that focus on specific issues or aspects of health care and clinical practice.

They consist of measuring a clinical outcome or a process, against well-defined standards set on the principles of evidence-based medicine. Aim of the audit is to highlight the discrepancies between actual practice and standard in order to identify the changes needed to improve the quality of care. A peculiar characteristic of the clinical audit is the “professionalism” of the initiative, which is expressed by some typical ingredients: clinical specific competence of the participants, the confidentiality of the results, the object strongly connected to the “quality” of professionals. From a methodological point of view, clinical audit consists of a “quality loop” (Figure 1): once chosen a topic and set shared and measurable criteria and standards, current clinical practice is evaluated, especially in terms of process or outcome, and suggestions for improvement are developed and applied, and then the cycle can begin again[2].

Figure 1.

Clinical audit cycle.

The audit should not be confused with data collection activities (i.e., benchmarking) or clinical research: the latter, in fact, aims to define the characteristics of good practice on a unknown land, while the audit compares the current practice against well-defined and established standards[3]. The final aim of the clinical audit is always improving the care provided to the patient.

This achievement may be reach through different actions: (1) Increase the culture of clinicians; (2) Solve a problem; (3) Reduce the variability of professional conduct (standardize); and (4) Reduce the gap between theoretical standards and real life.

PRINCIPLES OF THE CLINICAL AUDIT

Step 1: Preparing for the audit

Good preparation is crucial for the success of an audit project.

The key elements to design valuable clinical audits are: choosing the topic, defining a clear purpose and providing the necessary organisation in terms of audit staff and resources.

The first step that must be accomplished in designing a clinical audit is to identify the topic (Table 1). The topic of the audit can be loosely identified in clinical practice and may relate to the adequacy of a care process or that of the results[4]. An audited theme should have specific characteristics: it should be of great clinical importance, of easy collection and analysis, and source of important consequences. The personnel involved in the audit have a key role in setting priorities among clinical problems to deal with. By choosing a suitable theme various aspects should be considered.

Table 1.

Factors to consider in the decision on a topic for a clinical audit

For the choice of an appropriate theme for a clinical audit, assess that:

The problem to be audited has an important impact in terms of costs, resources, or risk

There is some strong scientific evidence available (guidelines, systematic reviews)

The improvements made on the subject in question can be easily evaluated and source of important clinical/organisational consequence.

In particular, it would be a good choice to face a problem that involves the clinician in terms of: (1) High volumes of work; (2) High costs in terms of health and/or economic; (3) High risk; (4) High variability; (5) High complexity; and (6) High innovation.

Rare events, such as complex clinical cases or sporadic adverse events, are not an appropriate topic for a clinical audit, and should be analysed with more adequate methodologies (i.e., Root Case Analysis)[5]. Once the topic has been selected, the purpose of the project must be defined, so that a proper audit methodology can be chosen and designed.

The aim of an audit project could include the implementation of new processes (for example laboratory protocols, surgical procedures, etc.) and/or the improvement of current strategies[6].

Moreover, before beginning a clinical audit, organisations should clearly declare the resources allocated to support the project management (data collection, hardware and software required) and for the training of the clinical staff, including education on clinical audit techniques, facilitation and data management[7,8].

Regarding the audit project team, it is advisable that it be customised for the specific audit project, with team members providing many of the skills needed. For example, if the topic of the audit is the management of vascular access in patients undergoing haemodialysis, it will be useful to include nephrologists, vascular surgeons and dialysis nurses in the audit team[9].

Step 2: Selection of indicators, criteria and standards and definition of intervention strategies

Once the preliminary issues of the audit have been defined, the next step is to set the standards, which the current clinical practice will be compared to. At this point, it is important to clarify some definitions: (1) Indicator: a variable that allows to describe complex phenomena and to measure changes in relation to defined criteria, in order to guide the decisions aiming at obtaining or maintaining the changes. It can be expressed as absolute number, percentage, rate, or average; (2) Criterion: it is a definable and measurable aspect of health care that describes its quality. The audit criteria are explicit statements that define an outcome to be measured. In a clinical audit, it is a declaration of what should happen on the basis of good practice, and it should be evidence-based[10]; and (3) Standard: it is the standard of care to be achieved for each specific criterion, usually expressed as a percentage. It represents the threshold of acceptability, that is, the value that defines the upper or lower limit, so that the quality of care is considered to be appropriate[11]. Some indicators are so important that the standards must be achieved in 100% of patients (e.g., use of masks during the dressing of central venous catheters), but in general it is sufficient to meet the standard in a lower percentage (for example, in 80% of patients)[4].

The choice of criteria and standards is one of the most critical points in the design of a clinical audit and it requires the collaboration of all participants in the audit. Indeed, the quality of care provided (i.e., the final result of the audit) will be evaluated just on the basis of a comparison with these parameters.

The sources where criteria and standards can be drawn from may be: international guidelines, scientific literature, expert consensus, data obtained by other health care facilities and personal case studies[12-14]. The stronger the evidence taken as a reference will be, the more the results of the comparison with daily clinical practice will be reliable. However, to design an effective clinical audit, it is important that the standard and criteria be shared with colleagues prior to the review of the collected data, since they should not be object of rearrangement in the course of verification, nor be changed retrospectively, in the light of the findings derived from the audit itself.

Finally, the audit team should also define the intervention strategies to be implemented in case of important discrepancies between standards and actual clinical practice. These strategies should be discussed, shared, clear and easy feasible according to a structured algorithm.

Step 3: Data collection

In clinical audit data can be collected prospectively or retrospectively[15]. Taking into consideration past clinical documentation, the latter method is certainly faster, but often the quality of the collected information is not optimal.

Perspective audits are more expensive in terms of time, but they allow a more accurate design, while offering a more realistic description of the current clinical practice. Before proceeding with data collection, it is necessary to carefully plan the variables to be recorded, and define the type of analysis to be conducted on the collected data. These points are important to prevent the collection of useless data or, conversely, the lack of essential information. A specific-designed form or a database should be arranged to collect patient records[16].

Moreover, it may be appropriate to carry out a sampling (preferably using randomized methods) if there is a very large number of patients to be examined, also in relation to the degree of confidence that one wants to achieve and the resources actually available (time, money, personnel)[17].

Collected data can be quantitative or qualitative, such as interviews, questionnaires or comments and data sources can be various, including medical records, results of biochemical and instrumental evaluations and/or other different archives[18,19]. The medical record is certainly the main source of information, but it is often incomplete. In this regard, highlighting the inadequacies of data management, already in the preliminary phase of data collection, the audit improves the existing information flow. Finally, it is worth pointing out that in every moment of data collection and analysis, patient privacy must be protected, making the information collected anonymous and explaining the reasons for the data collection, in case of direct involvement of patients themselves[20].

Step 4: Comparison of collected data with the standards and development of corrective actions

This is the central phase of clinical audit. In this phase, the team of professionals interested in the audit analyses the data and compares them with the pre-set standards. It is important to note that the critical nature of this moment lies in the fact that the professionals involved in the audit process can interpret the audit as an inspection of their clinical activity, thus becoming, unconsciously, an obstacle to an effective data analysis (Table 2)[21].

Table 2.

Facilitating factors and barriers for effective clinical audit

Facilitating factors	Obstacles
Clarity of design and data collection	Not clear objectives and planning
Good planning	Lack of resources-heavy workload
Organisation support	Lack of clarity on the method
Dedicated staff	Lack of organizational support
Collective analysis of the results	Unwillingness to change

For this reason, the meeting where the results of the audit will be discussed must be carefully prepared, paying particular attention to all aspects of communication and social skills[22,23].

Moreover, these contents must be pre-emptively shared with those who have proposed the audit. From the comparison of actual data with the theoretical standards different results might emerge, and the standard could be reached or not. In the event that the standard is not met, it should be assessed whether or not there is the possibility of a real improvement. In fact, if the data are not in line with the standards but they are sufficiently close, one might decide that any further improvement is difficult to achieve, and therefore it would be useful to invest resources in the assessment of other problems. In the case there is a significant difference between information gleaned from the clinical documentation and standards, collegial discussion should highlight the barriers to the achievement of the standard[24]. Afterwards, audit methodology requires that the audit team elaborate intervention strategies and recommendations, according to the indications preliminarily set[25]. Such advices or recommendations should take into account organizational factors (in terms of economic resources, timing, dedicated staff) and the context in which the audit takes place. For this reason it is imperative that the developed recommendations be clear, explicit and shared[26]. The mere dissemination of educational materials, such as guidelines, has little effect if they are not accompanied by selected methods of implementation, such as training seminars or discussions among peers[27].

Instead, in case the results obtained from the audit can be considered satisfactory, it is equally indispensable to provide a form of monitoring. Finally, all the findings drawn from data analysis and the subsequent discussion, including strategies to implement change, should be reported in a detailed account to be distributed to all participants of the audit, as feedback and reminder of the work done.

Step 5: Check and maintenance of improvements

The audit cycle ends with the stage of verification and monitoring of implemented strategies[2,4].

Indeed, it is essential for a proper process of clinical audit to schedule periodic verifications of the effects of the changes introduced. It would be advisable to use a data collection and an organizational strategy similar to that used for the previous analysis, so that the results are comparable.

If it emerges that the objectives have not been achieved and the plan of improvements was not effective or sufficient, it could be necessary to make changes to planned strategies.

However, also in case of success, a monitoring plan should be equally scheduled in order to maintain the improvements made.

EFFICACY OF THE CLINICAL AUDIT

There is conflicting evidence on the effectiveness of clinical audit[28]. A systematic review of the Cochrane Study Group has considered 140 studies in which clinical audit and the corresponding feedback were tested alone or in comparison to other types of interventions (meetings, distribution of printed materials, etc.). In the studies included in this review, the results produced by the audit were widely variable, from a negative to a very positive effect. When the audit was effective, the effects generally ranged from small to moderate. The review concluded that the relative effectiveness of an audit is likely to be greater when baseline adherence to recommended practice is low and when feedback is carried out with greater intensity[29]. Therefore, at the moment, scientific evidence does not provide clear support about the real effectiveness of clinical audit. This finding could be a starting point to design studies and analyses to validate clinical audit in different operating contexts[30].

CLINICAL AUDIT IN NEPHROLOGY

Medical literature offers several studies on audits conducted in the field of clinical nephrology, especially in patients on haemodialysis (HD). The reported studies have evaluated different aspects of organizational management and clinical research, such as the problems associated with late referral, vascular access, the management of hypertension and anaemia[31-33]. A careful analysis of these studies shows that the research has been mainly focused on the comparison between data collected from several case studies and indications of the guidelines. Therefore, the majority of these studies lack in the processes of cyclicity and verification that, as aforesaid, are the distinctive and characteristic features of clinical audit. An example of a well-conducted audit has been reported in a paper of an Australian group that has performed an audit in order to assess the effect of a multi-disciplinary intervention on the choice of dialysis vascular access, aiming at reducing the use of central venous catheters[34]. The first data collection on 184 incident dialysis patients was useful to recognize the problems in limiting the use of arteriovenous fistula, such as communication difficulties with patients or organizational shortcomings. Then, basing on the difficulties identified, the audit team developed specific intervention strategies (i.e., promotion of educational skills, facilitated access to the operating room, direct nurse involvement, etc.), that resulted, 12 mo later, in a significant increase in the number of patients starting dialysis with an arteriovenous fistula (75% vs 56 % of control baseline, P < 0.01).

Many audit projects have been also focused on management of hypertension in HD patients and different aspects have been investigated, such as the role of sodium dialysate concentration and dialysate temperature in the determining blood pressure (BP) levels[35,36].

Interestingly, in a recent study we tested whether a clinical audit in se is effective in improving BP control in a population of patients on regular HD.

We studied 177 adult prevalent HD patients, recording data on factors affecting BP and anti-hypertensive drug regimen at months -1 (Pre), 0 (the date of the audit- Audit), and +1 and +6 after the audit.

Hypertensive patients were identified, cases were discussed and recommendations for improving BP management were recorded, and then returned to each physician as a reminder and a feedback of the audit process.

The interventions included the reduction of extracellular fluid volume in patients with fluid overload, use of interdialytic ambulatory blood pressure monitoring and bioimpedance, initiatives aimed to increase patient compliance and modulation of dialysis sodium content or temperature. Interestingly, the announcement of the audit by itself was associated with a decreased prevalence of hypertension (Pre 64.4% to Audit 58.7%) and a further decrease followed the audit (Post-1 51.1%, Post-6 47.6%, P < 0.05 vs Audit). Systolic BP in hypertensive patients also decreased (mean decrease was -8.5 and -14.1; P = 0.007 and P < 0.001 at Post-1 and Post-6), being also associated with a reduced number of drugs assumed, thus proving that clinical audit is an effective tool to improve BP control in HD patients[37].

Mineral metabolism disorders in Chronic Kidney Disease (CKD-MBD) are an example of a suitable topic for a clinical audit. Indeed, they are common in HD patients and are associated with a number of clinical symptoms and complications, including cardiovascular diseases[38].

However, although MBD in HD patients are the object of intense research activity, their prevention and treatment still remain unsatisfactory[39]. In this view, we performed two large multicentre audits aiming to enlighten the obstacles that hamper the successful control of MBD by a straightforward “patient-oriented” approach[40,41].

Overall, we collected information and discussed the cases of about 700 prevalent HD adult patients according to the audit methodology.

First of all, we confirmed the data regarding the difficulty to achieve therapeutic targets, showing that only 15%-20% of the evaluated patients presented Ca, P and PTH values simultaneously controlled[42].

Then, evaluating the factors related to unsatisfactory results, we found that low compliance with treatment was the major determinant of failure (43.5% of the cases).

However, we observed a discrepancy between the analysis of factors accounting for therapeutic failure and the interventions planned. In fact, while the low compliance was recognized as the main cause of therapeutic failure, most of the interventions were focused on pharmacological therapy. Consequently, six months after the audit we found that, against a significant increase in the amount of drugs prescribed, the control of MBD parameters did not improve.

Therefore, the results of the audit suggested that low compliance with treatments is a main but still neglected cause of failure in the achievement of MBD control in HD patients, while increase of drug administration, regardless the awareness to the compliance to the therapy, is insufficient to obtain an overall satisfactory rate of therapeutic success.

This finding is particularly important, since indicates that future therapeutic strategies, beyond the development of new drugs, should include the implementation of feasible educational programmes addressed to both health personnel and patients. This kind of study shows the potentiality of a clinical audit that allows to effectively compare theoretical standards with daily clinical practice, providing suggestions to improve quality of care.

FUTURE APPLICATIONS

Audit methodology could be potentially extended to several other issues in the setting of clinical nephrology.

For example, it could be useful to evaluate the causes of treatment failure in patients undergoing peritoneal dialysis, such as to implement protocols to reduce the rate of central venous catheter-related infections. Moreover, clinical audit could be a feasible tool to solve organizational problems, such as the delays on the waiting list for kidney transplantation.

Finally, a clinical audit could be used to face more general topics, which may involve also renal patients, such as management of dyslipidaemia (for example, evaluating the appropriateness of statin prescription) and implementation of lifestyle change.

CONCLUSION

Quality control, and consequently the right allocation of resources, is becoming a central issue in the management of Health Care Systems. Several tools are deployed to provide a monitoring of the levels of care and improve its quality. Among them, clinical audit is one of the most popular and widespread. In the specific field of clinical nephrology, this method has proven its effectiveness in facing different problems, such as hypertension and mineral metabolism control. However, it still seems necessary to spread the understanding of clinical audit and promote its systematic application both nationally and locally, so that it can be part of the expertise of each health care provider, together with other quality improvement techniques. In Table 3 we present a checklist for the planning of a clinical audit.

Table 3.

Checklist for the planning and validation of a clinical audit

Item	Yes/ No
Promoting a clinical audit	The audit topic has been decided according to the needs of the working group.
	The objectives are clearly specified.
	Indicators, criteria and reference standards have been set according to literature, guidelines and/or the consensus among experts.
Design and planning	The audit has been organized in different stages and times, assigning specific responsibilities.
	Necessary resources have been allocated.
	The population/reference sample has been defined.
	Tools for data collection have been designed, preliminarily defining data management methods.
	The whole material has been proposed in advance to the participants.
Data collection	Those who participated in the preventive phase have been involved.
	The established phases have been met.
	Data have been correctly collected.
Data analysis Interventions	The results have been discussed with the participants to the audit and other interested parties.
	A structured strategy to implement changes has been defined.
	Written reports of the results have been made and sent to all the participants.
Checking the audit effectiveness	A check of the effectiveness of the changes introduced has been planned.
	The verification has been formally documented.

ACKNOWLEDGMENTS

We thank Marina Nazzaro for the English editing.