How to conduct External Quality Assessment Schemes for the pre-analytical phase?

Abstract

In laboratory medicine, several studies have described the most frequent errors in the different phases of the total testing process, and a large proportion of these errors occur in the pre-analytical phase. Schemes for registration of errors and subsequent feedback to the participants have been conducted for decades concerning the analytical phase by External Quality Assessment (EQA) organizations operating in most countries. The aim of the paper is to present an overview of different types of EQA schemes for the pre-analytical phase, and give examples of some existing schemes. So far, very few EQA organizations have focused on the pre-analytical phase, and most EQA organizations do not offer pre-analytical EQA schemes (EQAS). It is more difficult to perform and standardize pre-analytical EQAS and also, accreditation bodies do not ask the laboratories for results from such schemes. However, some ongoing EQA programs for the pre-analytical phase do exist, and some examples are given in this paper. The methods used can be divided into three different types; collecting information about pre-analytical laboratory procedures, circulating real samples to collect information about interferences that might affect the measurement procedure, or register actual laboratory errors and relate these to quality indicators. These three types have different focus and different challenges regarding implementation, and a combination of the three is probably necessary to be able to detect and monitor the wide range of errors occurring in the pre-analytical phase.

Introduction

In laboratory medicine, several studies have described the most frequent errors in the different phases of the total testing process (TTP) (1–12), and a large proportion of these errors occur in the pre-analytical phase (2,5,13–17).

The first step in improving the quality of the pre-analytical phase is to describe potential errors and to try to estimate which errors are most dangerous for the outcome of the patient (13,18–22). Existing pre-analytical procedures should be compared to existing recommendations and thereafter improved to minimize the risk of errors. In addition, the frequency of errors should be recorded on a regular basis to detect improvement or deterioration over time, and further to explore if procedures should be changed.

Schemes for recording of errors and subsequent feedback to the participants have been conducted for decades concerning the analytical phase by External Quality Assessment (EQA) organizations operating in most countries. It is reasonable that these organizations also take upon them to set up EQA Schemes (EQAS) for the pre-analytical phase. At present, however, most EQA organizations do not offer such schemes. An important challenge, when developing EQAS for the extra-analytical phases, is the variety of locations and staff groups involved in the total testing process, of which several are outside the laboratory’s direct control. Test ordering, data entry, specimen collection/handling and interpretation of results often involve other than laboratory staff. Some of the pre-analytical schemes, e.g. schemes on test ordering, could also involve other health care professionals. The contact and communication between the clinicians/nurses and the laboratory staff is often limited (7,23). The ISO 15189: 2012 states that “External quality assessment programs should, as far as possible, provide clinically relevant challenges that mimic patient samples and have the effect of checking the entire examination process, including pre- and post-examination procedures” (item 5.6.4) (24). To our knowledge, and in contrast to what is required for the analytical phase, accreditation bodies do not ask laboratories for results from EQAS regarding the pre-analytical phase. If this is changed, it will be a driver for the EQA organizations to set up such schemes, and for the participants to use them. Consequently, a joint effort between EQA organizations, accreditation bodies and clinical laboratories seems necessary to implement pre-analytical EQA schemes. The aim of this paper is to present an overview of different types of EQA schemes for the pre-analytical phase and to give examples of existing schemes.

How to perform a pre-analytical EQA scheme?

Fortunately, EQAS for the pre-analytical phase are increasingly being developed, and roughly three different methods have been conducted:

• Type I: Registration of procedures: Circulation of questionnaires asking about procedures on the handling of certain aspects of the pre-analytical phase, e.g. how “sample stability” or tube filling are dealt with, which criteria are used for sample rejection, and how these issues are communicated to the requesting physicians.

• Type II: Circulation of samples simulating errors: For example distribution of real samples with matrixes or samples with contamination which might interfere with the measurement procedures. This is similar to usual analytical EQAS, since a sample is sent to the laboratories. Case histories can be sent together with the sample to elucidate how these samples are dealt with, and how the results are communicated to the physicians.

• Type III: Registration of errors/adverse events: The incidences of certain types of pre-analytical errors (some of them could be quality indicators (QIs)) are registered and reported by the laboratories to the EQA organizer.

The EQA organizer should as usual provide feedback reports for the participating laboratories, enabling the laboratories to compare their results with the other participants. In addition, the feedback reports should include advice on how to minimize errors. In case of error reporting (Type III), the EQA organizer should also lead the harmonization of QIs for a valid comparison of error rates between different locations. Examples of the three different methods to conduct pre-analytical EQAS will be given below. Ongoing EQAS are summarized in Table 1.

Table 1.

Examples of ongoing pre-analytical EQAS.

Method	Pre-analytical issues studied	Frequency	Performed by
Type I. Registration of procedures
Registration of procedures via web-based multiple choice questionnaire	Clinical chemistry: Hemolysis, stability of samples (2011) Hemostasis testing: Phlebotomy, sample handling and sample acceptance (2012) Glucose: Sample handling and sample treatment (2013)	1 × year	Norwegian Clinical Chemistry EQA program (NKK) Not published
Registration of procedures via web-based multiple choice questionnaire	Hemostasis testing	2 × year	ECAT/INSTAND 2011 (25)
Clinical case-based European EQAS covering pre-analytical, analytical and post-analytical phase. Five sets of multi-specimen samples	Porphyria: Case history based test ordering	2 × year	Norwegian Porphyria Centre (NAPOS)/The European Porphyria Network (EPNET (26)
Registration of procedures via web-based multiple choice questionnaire	5 general pre- and post-analytical questions, 5 questions within specific disciplines (i.e. coagulation, hematology, immunology, microbiology)	2 × year	* Quality Control Center Switzerland (CSCQ)
Registration of procedures via multiple choice questionnaire	Urine chemistry, clinical chemistry	4 × year	* WEQAS
Registration of procedures via web-based multiple choice questionnaire	Hematology, sample handling	1 × year	* INSTAND
Type II. Circulation of samples simulating errors
Circulate samples for extraction of RNA/DNA	Sample preparation for DNA and RNA testing	1 × year	SPIDIA-DNA, 2012 SPIDIA-RNA, 2011 European Commission (EC) (33, 34)
Circulate samples	Sample indicies – lipemic, icteric, hemolysis index	4 × year	* WEQAS
Type III. Registration of errors/adverse events
Q-Track (since 1998, 1 × year, ongoing) programs, registration of error rates	Patient/sample identification, specimen handling/preparation, specimen acceptability, customer satisfaction	4 × year	College of American Pathologists (CAP) (39)
Registration of rejection of samples	Registration of the rejection rate and causes for rejecting the samples during 1 month or 100 rejections	2 × year	Committee for the Quality of the Extra-analytical phase (started within The Spanish Society of Clinical Chemistry and Molecular Pathology (SEQC) in 1998 (41)
Registration of key incidents which represent either the most frequent or most serious incident	Patient identification, incorrect patient preparation, phlebotomy, sample preparation/handling and sample acceptability	4 × year	Key Incident Monitoring and Management Systems Quality Assurance (KIMMS QA) 2009 (43).

^*Personal communication at the EQALM meeting in Bucuresti in 2013 where the participants were asked if they had any pre-analytical EQAS. Three EQA organizers replied: WEQAS (Annette Thomas, http://www.weqas.com/eqa/index.html) INSTAND (http://www.instandev.de), and CSCQ (Dagmar Kesseler, http://www.cscq.ch)

Type I: Registration of procedures

Identification and registration of pre-analytical procedures can be performed by circulating questionnaires, with open or multiple choice questions. Case histories to illustrate the potential consequences of pre-analytical errors for the patients may be included. A clear advantage of this type of EQAS is the limited resources needed for distributing and completing the scheme. Several aspects of the pre-analytical phase can be covered, and the questionnaire may potentially reach laboratories all over the world in a short time using an electronic contact and report form.

The Norwegian EQA program (NKK) has developed a pre-analytical EQA scheme which aims to identify especially problematic pre-analytical issues related to clinical chemistry analyses and hemostasis testing. This scheme is carried out by circulating web-based questionnaires (mostly multiple choice questions) to the Norwegian laboratories once a year dealing with different aspects of the pre-analytical phase. The laboratories receive a feedback report after each survey, showing their own results, the overall results and an overview of the relevant recommended procedures from guidelines and recent studies. The three surveys performed so far (2011, 2012 and 2013) (Table 1), concluded that there is a large variability in several pre-analytical practices and considerable room for improvement.

In 2009, the Croatian Chamber of Medical Biochemists (CCMB) performed a survey in Croatian laboratories to investigate appropriateness of procedures in the extra-analytical phases and to detect procedures most prone to errors of potentially clinical importance (27). A multiple choice questionnaire including pre-analytical conditions and criteria for sample acceptance and procedures of phlebotomy were circulated to members of CCMB. The study concluded that there is an urgent need for improving activities by appropriate recording and monitoring of the extra-analytical phases; emphasizing the importance of initiating pre- and post-analytical EQAS.

Labquality, the Finnish EQA organization, are about to start pre-analytical EQAS in 2014. The program will include case studies and multiple choice questionnaires concerning phlebotomy practices, pre-analytical procedures for clinical chemistry, micro-biology and for blood gas analyzers (Personal information from Jonna Pelanti, Labquality, FI).

Challenges regarding registration of procedures as pre-analytical EQAS

To avoid misinterpretations, the questionnaires used in EQAS should be validated by experts in the field and/or in pilot studies to a few laboratories. This is especially important if the questionnaires are circulated in different countries and not translated into local languages. The questions should be designed to stimulate the participants to answer how the procedures really are performed, and not how they ideally should be performed (e.g. ask for written procedures). Providers of such schemes should also be aware that queuing of answers (multiple choices) might influence the results. Standardized follow-up of the surveys should be performed by offering advises to the participants on how to improve their procedures.

Type II: Circulation of samples simulating errors

EQAS for the pre-analytical phase can be performed using real samples with a matrix potentially interfering with the measurement procedures, as for example hemolysed, lipemic, or icteric samples or samples containing drugs known to interfere. Another example is to send the wrong sample material (e.g. serum instead of plasma). This type is in some instances comparable to analytical EQAS, since a sample is sent to the laboratories for measurement of pre-defined analytes. As for analytical EQAS, case histories can be included to elucidate which pre-analytical procedures are performed and how the results (or lack of results in case of sample rejection) are communicated to the physicians.

The feedback reports for these surveys are comparable to feedback reports for regular EQA surveys, and should include a comparison of the laboratory’s results to the results of all participants in addition to an overview of existing guidelines/recommendations and recent publications.

Surveys focusing on problematic areas within the pre-analytical phase may not be suitable for regular schemes, but can be conducted as specialized pre-analytical EQAS performed once (or a couple of times). In this way, the EQAS organizer can choose to study the most relevant issue present. The Nordic committee for External Quality Assurance Programmes in Laboratory Medicine (NQLM) carried out four different EQA interference surveys between 1999 and 2002 (28). The aim was to assess the effects of hemolysed, icteric and lipemic samples on some common clinical chemistry serum analyses. During 2014, a similar EQA survey will be performed in the Nordic countries to receive updated information on analytical performance and handling of hemolysed samples.

One of the major goals of the newly started European Commission funded project SPIDIA (Standardization and improvement of generic pre-analytical tools and procedures for in-vitro diagnostics) has been to develop evidence-based guidelines for the pre-analytical phase of blood samples used for molecular testing (29). The SPIDIA project has resulted in two pan-European EQAS focusing on the pre-analytical phase of DNA and RNA-based analyses (30,31) (Table 1). Blood samples were sent to the laboratories which performed DNA/RNA extraction of the material, the extracted material was returned, and the EQA organizer assessed the quality. The results of these surveys will provide the basis for performing new pan-European EQAS and further provide the basis for implementation of evidence-based guidelines for the pre-analytical phase of DNA and RNA analyses (29).

Challenges regarding the use of real samples in pre-analytical EQAS

The requirements for samples to be used in pre-analytical EQAS are similar to the requirements for samples in analytical EQAS. The samples should be similar to native patient samples containing the actual interferences, and should be homogenous and stable during the survey period. Additionally, they should reflect actual poor pre-analytical conditions like incorrect blood sampling and sample errors due to using tubes containing incorrect additives (e.g. serum or EDTA when citrated plasma is required) or inappropriate specimen preparation, centrifugation, aliquoting, pipetting or sorting. Mimicking real life situations is a challenge when producing sample material in a large scale. Only a limited set of pre-analytical problems can be investigated by utilizing this method (Type II), and for some areas it seems natural to perform the EQAS once or on an irregular basis. One should also take into consideration the bias introduced if the participants know they are to receive a sample with an interfering substance (participating in a pre-analytical scheme), and it might therefore be better to contaminate samples in a regular EQA scheme.

Type III: Registration of errors/adverse events

EQAS for the pre-analytical phase can be performed by initiating registration of defined and standardized types of errors (or adverse events). The error rate (or the rate of adverse events) during a pre-specified time interval should then be reported to the EQA organization. Systematic registering of incidents may point out bottle necks and indicate which areas are most error-prone (32). An accredited laboratory must have a system for registration of incidents and are required to establish QIs as a measure of the laboratory’s performance (ISO 15189) (24). Utilizing this system in an EQA program (e.g. the EQA provider suggest QIs to the laboratories; the laboratories thereafter report their QI data to the EQA provider, making comparison between laboratories regarding QI performances possible) is perhaps the best EQA program for monitoring pre-analytical (and post-analytical) errors. The QIs found most appropriate should also be harmonized between EQA organizers.

The participants should receive a feedback report showing their own QI performance compared to the results of all participants and to desirable quality specifications. It should also include historical data showing the development of the performance of the laboratory’s QIs.

In 1989, the College of American Pathologists (CAP) started the Q-Probe program, carried out as time-limited studies lasting 1–4 months, to define critical performance measures in laboratory medicine, describe error rates on these measures and provide suggestions to decrease the error rate (33,34). In 1998, CAP developed this concept further and started the ongoing Q-Track program (33) (Table 1). Data is collected according to defined methods and time frames by utilizing standardized forms. CAP provides a quarterly “Performance Management Report” package which helps the laboratories to identify issues providing the opportunity for improvement and to monitor the effect of changes. So far, information on error rates from more than 130 worldwide inter-laboratory studies has been included in CAP’s databases (34). The error rate has been determined for almost all the important steps of the total testing process, QIs have been established, and suggestions for error reduction provided. Improvement can be quantified as exemplified by one of the most frequently occurring errors; missing wristbands, where the median error rate was reduced from more than 4% to less than 1% during four years (35). Similarly, participants who participated for two years demonstrated a significant declining trend in sample rejection rate (36).

The Spanish Society of Clinical Chemistry and Molecular Pathology (SEQC) has since 1998 conducted two EQAS per year, registering rejections of samples and the causes for rejections (Table 1). A retrospective analysis of 10 surveys (2001–2005) summarized and evaluated the data and concluded that “specimen lost or not received” (37.5%), “hemolysis” (29.3%) and “clotted samples” (14.4%), were the main causes for sample rejection (37). The percentage of sample rejections was highest if the samples were collected in wards outside the central laboratory. The retrospective evaluation of the program resulted in a simplified scheme as some of the included variables (e.g. type of serum tubes (gel, silicone or no separator), type of anticoagulant employed (liquid or solid), extraction procedure (with or without vacuum) and material employed (glass or plastic device)) turned out to be irrelevant.

The results from an EQAS performed by Australian chemical pathology laboratories, aiming at measuring transcription (defined as any instances where the data on individual request forms were not identical to the data entered into the laboratory’s computer system, e.g. patient identification, patient sex and age, patient ward location or address, tests requested and requesting doctor identification) and analytical errors, have been summarized (38). This study showed a large inter-laboratory variation in total error rates (5 to 46%). The maximum error rates detected were 39% and 26% for transcription and analytical errors, respectively. The study concluded that there is a need to establish broader quality assurance programs and performance requirements for the pre-analytical phase.

Several pilot EQA-schemes were performed during 2006–2009 throughout Australia and New Zealand, and the experiences from these surveys were used to make the current “Key Incident Monitoring and Management Systems Quality Assurance” (KIMMS QA) program (39) (Table 1). In this scheme, the participating laboratories are asked to register a subset of pre- and post-analytical (PAPA) incidents, which represents either the most frequent or the most serious incidents, or which represents incidents with the greatest opportunity for inter-laboratory benchmarking and improvement. The data from the participants is pooled to form a national frequency distribution of PAPA incidents, and each participating laboratory can compare their own results with this distribution. For 70% of all reported incidents, the root causes require interaction between the laboratory staff and other health care workers. In the last KIMMS EQA survey, the overall PAPA incident rate was 1.22% and the most frequent incident was inadequate patient or sample identification (0.28%) (39).

As a part of a project to reduce errors in laboratory testing, the IFCC Working Group on Laboratory errors and patient safety (WG-LEPS) aimed to develop a series of QIs, specially designed for clinical laboratories (40,41). The aim of the project is to provide a common framework and to establish a set of harmonized QIs which should cover all steps of the TTP. The group concluded that a model of QIs managed as an EQAS can serve as a tool to monitor and control the pre-, intra- and post-analytical activities. The QIs suggested by WG-LEPS were evaluated in a study performed during 2009 and 2011. The results showed that QIs in the analytical phase improved much more than the corresponding for the extra-analytical phases, probably because improvement in the extra-analytical phases may be more complex requiring close cooperation between the laboratory staff and the health care workers outside the laboratory (42). This finding indicates that measurement of errors alone will not reduce the error rates, but corrective actions which include cooperation, teamwork and firm follow-up on achievements are necessary.

In a recent study performed in a group of clinical laboratories in Catalonia (Spain), the results obtained for the QIs of key processes were analyzed over a five years period (2004–2008) (43). The objective was to determine the QI evolution, identify processes requiring corrective measures, and obtain robust performance specifications. Different indicators were evaluated according to efficiency and safety. The average yearly inter-laboratory median value for the different indicators over the five years period was used as the desirable specification for the indicators. The values obtained were transformed to the Six Sigma scale and processes with sigma values ≥ 4, were considered well controlled (44). The medians for most QIs were considered stable during the study period, however, if the sigma value was less than 4 the QIs were considered robust but subject to improvement. Examples of pre-analytic processes outside the laboratory that could be improved were “Total incidences in test requests” (sigma 3.4) and “Patient data missing” (sigma 3.4). Within the laboratory “Undetected requests with incorrect patient name” obtained the lowest sigma value (2.9). The method used in this study could also be performed as a regular EQA scheme for the pre-analytical phase.

Challenges regarding the use of QIs managed as an EQAS

The EQA organization should develop QIs and distribute registration systems to the laboratories, and this may be a quite labor intensive process. Harmonization is of uttermost importance, since variation in the methods used to register the errors might influence the results and make comparison between laboratories difficult. Harmonization of QIs may be difficult on an international basis if laboratory practices differ between regions, but harmonization within local areas should be possible. A challenge for the participating laboratories is the extra resources (time and personnel) needed to register their own errors. It is extremely important to focus on improving the system and not focus on each individual, since this might lead to under-reporting of errors. Most errors in laboratory medicine are classified as blameless and represent the accumulation of a number of contributing events, indicating the need to modify systems as opposed to disciplining individuals (32,45).

Conclusion

So far, very few EQA organizations have focused on the pre-analytical phase, compared to the analytical phase, even if the pre-analytical phase is more prone to errors. In this paper, three different types of pre-analytical EQAS are described, having somewhat different focus and different challenges regarding implementation. A combination of the three is probably necessary to be able to detect and monitor the wide range of errors occurring in the pre-analytical phase. It might be a good idea to start with pre-analytical EQA schemes nationally since they are easier to adapt to local conditions and results could be discussed on local users meetings. Based on the results of these schemes one could in co-operation with other countries develop international schemes aiming at harmonizing pre-analytical guidelines and QIs. Pre-analytical EQA schemes of type II will in many instances be size-restricted by the supply of EQA sample material, while type I and III might be more suitable for international cooperation through international EQA organizations (e.g. EQALM). Development of pre-analytical EQA schemes and publication of the results should be encouraged since information of such schemes and their ability to improve pre-analytical routines in the laboratories are scarce in the literature.