GUIDELINE IMPLEMENTATION IN CLINICAL PRACTICE: USE OF STATISTICAL PROCESS CONTROL CHARTS AS VISUAL FEEDBACK DEVICES

Abstract

Objective:

To use statistical control charts in a series of audits to improve the acceptance and consistant use of guidelines, and reduce the variations in prescription processing in primary health care.

Methods:

A series of audits were done at the main satellite of King Saud Housing Family and Community Medicine Center, National Guard Health Affairs, Riyadh, where three general practitioners and six pharmacists provide outpatient care to about 3000 residents. Audits were carried out every fortnight to calculate the proportion of prescriptions that did not conform to the given guidelines of prescribing and dispensing. Simple random samples of thirty were chosen from a sampling frame of all prescriptions given in the two previous weeks. Thirty six audits were carried out from September 2004 to February 2006. P-charts were constructed around a parametric specification of non-conformities not exceeding 25%.

Results:

Of the 1081 prescriptions, the most frequent non-conformity was failure to write generic names (35.5%), followed by the failure to record patient's weight (16.4%), pharmacist's name (14.3%), duration of therapy (9.1%), and the use of inappropriate abbreviations (6.0%). Initially, 100% of prescriptions did not conform to the guidelines, but within a period of three months, this came down to 40%.

Conclusions:

A process of audits in the context of statistical process control is necessary for any improvement in the implementation of guidelines in primary care. Statistical process control charts are an effective means of visual feedback to the care providers.

INTRODUCTION

The quality of service in primary care is important in influencing and shaping patient satisfaction. It improves the delivery of care at this level and improves health outcomes.¹ The provision of clear, high quality evidence based guidelines and effective strategies for their implementation²,³ are necessary for any systematic, effective and efficient improvement in the quality of care in clinical practice.⁴ Failure to implement these guidelines, on the other hand, is associated with poor outcomes.⁵

In order to bring about change in clinical practice, it is necessary to generate local support for the implementation of guidelines, a vital part of development.⁶ Any obstacles to implementation should be properly dealt with⁷ systems of clinical audits carried out and information feedback provided to care providers.⁶,⁸,⁹

Short-term success in the implementation of guidelines may not guarantee continued provision of good care.¹⁰,¹¹ Therefore, it is necessary to put in place a dependable system of sustained improvement. Frequent serial audits, commensurate with the volume of work in the service concerned, as well as a simplified, systematic feedback given to those involved in care, can provide this system of sustained improvement.

Although audits and feedbacks are widely used in healthcare to improve professional practice,¹² analysis and interpretation of results of audits may be difficult unless methods used take into account the random variations inherent in any process.¹³ The use of scientific techniques that take such variations into account is important for the monitoring of clinical practice¹⁴ to detect the reaons for variations in performance in order to minimize them by making modifications to the service and¹⁵ providing a mechanism for continuous, concurrent evaluation of reality.¹⁶ According to given clinical contexts, these techniques include Sequential Probability Ratio Test¹⁴ and Statistical Process Control.¹⁷,¹⁸

In most primary care facilities, there is no system of regular, routine audits in the different service components. Any audits done are sporadic unrelated initiatives. Whatever assessments and reviews there are, often subjective with no explicit reference to predetermined standards of practice.¹⁹

In order to improve the process of care in different areas of practice, we conducted a series of audits focusing on clearly defined processes and standards. These were based on guidelines provided by parent departments and the feedback of results given to all employees involved in direct patient care.

This current series of audits was initiated as a result of observations made by colleagues in the pharmaceutical services on the lack of conformity by most clinicians to the given guidelines. Our objective was to induce the required behavioral change in clinical practice by providing regular feedbacks to care providers on the status of their practice using scientifically sound methodology that recognizes random variation.

In this paper, we present the results of audits on prescribing and dispensing practices in the main satellite of King Saud Housing Family and Community Medicine Center, one of the primary care access points of National Guard Health Affairs in Riyadh.

METHODS

In the main clinic, one of the three satellites for the center, three general practitioners provide outpatient services for about 3,000 patients. Each physician generates more than 300 prescriptions per weeky. Six pharmacists provide dispensing services for routine and urgent care facilities in the center.

For the purpose of these audits, compliance with all components of the guidelines of prescribing and dispensing provided by the pharmaceutical services of King Fahad National Guard Hospital was used as the criterion for appropriate prescription processing. The guidelines consist of 14 components: recording file number, patient's age, weight , diagnosis, and generic drug name; specification of dose, frequency, and duration; avoidance of non-standard abbreviations, absence of contraindications (including interactions), recording of dispensing pharmacist's and prescribing doctor's names, date of prescription and legibility of handwriting.

Concerns on the lack of weighting the non-conformities according to their seriousness and the involvement of more than one component of care i.e. general practice clinic and pharmacy were discussed. By a consensus of the physicians and the pharmacists, a decision was taken to use the whole set of instructions in the guidelines.

Conformity to the given criterion, a binary outcome variable subject to random and systematic variations, was studied by constructing p-charts using the formula[], with the center line at 0.25, upper control line at 0.49, and lower control line at 0.01, based on the initial parametric target of proportion of non-conformities not exceeding 25% of prescriptions, a figure that was arbitrarily determined in consultation with management.

Sample size was set at 30, as this number satisfied the condition for sufficiency of sample size in that both n(p) and n(1-p) … 30×0.25, 30×0.75, are above 5.²⁰ Based on the work flow and staff availability, it was decided that fortnightly audits should be carried out.

A questionnaire was prepared and a database created using Epidata.²¹ A decision on the appropriateness of prescription processing was generated automatically by the database program based on the responses provided to the 14 binary yes-no variables. Non-conformity to any of the given components would render a prescription “non-conforming to the criterion of appropriate processing.” The database was deployed in the pharmacy, and pharmacists were trained to enter data and analyse them using EpiInfo.²²

A simple random sample of 30 prescriptions was chosen every two weeks from a sampling frame of all prescriptions during the period, using a computer program. A questionnaire was filled for each prescription by a pharmacist; data were entered in the database and checked by another pharmacist for accuracy of information. Although information about individual performance of concerned employees was available, individually tailored feedback was not implemented. The only intervention consisted of an explanation of Statistical Process Control Charts and Pareto charts of non-conformities for the last audit to all care providers in a group and an exhibition of charts for them at the work place. This report is based on data generated from 36 audits carried out from September 2004 to February 2006.

DATA ANALYSIS

Proportions are reported as percentages. Exact binomial confidence intervals are used for the estimation of binary outcome parameters. Cuzick's test P-Value is reported for trend across ordered groups. Analyses were carried out with Epi-Info 6.04d²² and Stata Version 8.2.²³

RESULTS

At the end of the audit period, complete information on 1081 prescriptions was available for analysis. Initially, 100% of the prescriptions studied did not non-conform to the defined criterion of appropriate processing of prescriptions. A noticeable decline in non-conformities was observed as a result of regular weekly feedback of the situation to physicians (Table 1, Figure 1). Within a period of three months, the proportion, although still not in control at the given level of 25%, had fallen to a level that was in control at 40%, with a maximum run of consecutive points on one side of center line at three 3 (Figure 1).

Table 1.

Average number of non-conformities per prescription by audits at King Saud City Family & Community Medicine Center, National Guard Health Affairs, Riyadh (September 2004-February 2006)

Figure 1.

Attribute Chart (p-chart) for proportion of prescription nonconforming with the given guidelines, compared with Center Lines at 0.25 and 0.4

Regarding the non-conformities, failure to write generic names (35.5%), patient weight (16.4%), pharmacist's name (14.3%), duration of therapy (9.1%) and the use of inappropriate abbreviations (6.0%) accounted for more than 81% of non-conformities (Figure 2). Mean non-conformities per prescription showed a significant decrease (p-value < 0.0001) over time, falling from 2.9 non-conformities per prescription at first audit to only 0.4 per prescription in the last audit (Table 1). Regarding individual non-conformities, recording the generic drug name was the most resistant to change (Figure 3), followed by the recording of the patient's weight. Recording pharmacist's identity, duration of therapy, and the use of non-standard abbreviations had improved much earlier. The first two, generic drug name and patients weight, were the most resistant to change probably because of the need for improvements in physicians’ knowledge, and in nursing services routines. A concurrent collaborative focus of continuing education might have produced quicker and better results.

Figure 2.

Pareto chart of Non-conformities with the given guidelines (N=1225)

Figure 3.

Attribute charts (p-chart) showing variation in improvement of individual non-conformities

DISCUSSION

Implementation of guidelines in healthcare has generally been reported as fragmented and inconsistent²⁴ and still remains a significant challenge for various healthcare organizations.^25–27 Various factors are responsible for this phenomenon. These include the lack of training of the care providers in quality management,²⁸ lack of resources, lack of awareness of the details of the guidelines, and the lack of acceptance of the given recommendations²⁵ by those involved in the process of care. The initial problems of learning to use statistical techniques at the work place may also be an obstacle, although experience tells us that this may not be such a daunting task.²⁹

As audits coupled with timely, individualized, and non-punitive feedback to care providers have resulted in better adherence to clinical practice guidelines,³⁰ a system of regular feedbacks of the status of practice to the care providers can be an effective, affordable, and scientifically sound tool that decision makers at different levels of administrative hierarchy can utilize to facilitate improvement.

A surprising initial 100% level of non-compliance was discovered in spite of the clear brief guidelines given to the different categories of care providers in the service more than a year before the process of audit was started.

In addition, other measures such as properly designed, recorded and archived prescription sheets, a well-systematized communication of pharmacists with physicians regarding their concerns about prescriptions, and an ongoing program of continuing medical education for various categories of staff are necessary.

This high proportion of non-conformity fell to 40% within a period of three months of regular audits and feedbacks. Our general feedback process consisted of the presentation of the p-charts and Pareto charts to a group of care providers without any individually tailored feedbacks of interviews. It is our feeling that any individual communication and counseling might have resulted in an even better outcome of achieving the given target of non-conformities of less than 25%.

The use of Statistical Process Control Charts as tools to detect and analyze process variation is a well-known technique. We found that, after we explained their structure to those involved in the process of prescribing and dispensing, the charts provided a very effective visual feedback by clearly depicting sample results compared with the targeted parameter.

The whole initiative was generated, managed, and maintained locally by physicians, nurses, and pharmacists, with no help from outside the practice. We strongly recommend incorporating such initiatives routinely in the functions of different components in primary care.

CONCLUSION

Ongoing audit processes should be incorporated in primary care routines. Statistical Process Control Charts put the audits in perspective and can provide a visual feedback of audit results to care providers in an effective, efficient and sustainable manner by means of local resource.