Evaluation of four quality indicators of the Pre-Analytical Phase External Quality Assessment Subprogram of the Fundación Bioquímica Argentina

Abstract

Pre-analytical phase external quality assessment programs contribute - through the interlaboratory comparison of quality indicators (QIs) - to the continuous improvement of the clinical laboratory total testing process. The purpose of the present work is to document the results derived from measuring four QIs within the framework of a pre-analytical phase external quality assessment subprogram in Argentina. The laboratories participating in this subprogram measured the following QIs: i) patients recalled for a new blood sample collection due to pre-analytical causes; ii) clotted samples from hemogram and coagulation tests; iii) clinical chemistry hemolyzed samples; and iv) requests with transcription errors entered into the laboratory information system. Results were expressed in percentage value and Sigma value. Databases were anonymized. A minimum acceptable quality level for the four QIs measured was recorded in the majority (75%) of the participating laboratories (Sigma > 3.0). It was nonetheless observed that the QIs of hemolyzed samples and requests with transcription errors entered into the laboratory information system deserve more attention. Through this pioneering experience in Argentina, the participating laboratories - some for the first time - could learn about their performance via interlaboratory comparison of results. This experience also proved to be motivating not only to improve the external assessment subprogram but also to continue working on the measurement of pre-analytical QIs for the continuous improvement of the clinical laboratory total testing process in Argentina.

Introduction

Over time, the concept of error in the clinical laboratory has evolved from a model focused primarily on the analytical phase to a model focused on errors that occur all throughout the clinical laboratory total testing process (TTP), including extra-analytical phases (1,2). The pre-analytical phase, in particular, is a stage that is not only key but also high-risk for patients as a result of its complexity and the variety of procedures and factors that it involves. Evidence has, in fact, been documented that errors in this phase represent up to 70% of the total errors in the TTP (3). For this reason, both internal and external quality control of the pre-analytical phase is of paramount importance to ensure clinical utility of the results issued by the laboratory (1,4). Risk analysis, systematic error detection and the implementation of pre-analytical quality indicators (QIs) are sine qua non requisites for an effective internal quality control (5). QIs have proven to be an effective tool to monitor processes as well as the efficacy of the corrective and/or preventive actions implemented, and therefore their measurement is required to comply with different clinical laboratory accreditation standard regulations (5-7). QIs are also objective tools with which it is possible to assess what happens in the laboratory during the TTP, thus allowing self- and inter-comparison among laboratories measuring the same QI (7). Still, the main recommendation is that, depending on their resources, laboratories should prioritize the most useful QIs for their processes to avoid unnecessary extra work that may limit the continuity and usefulness of their monitoring (5). By participating in pre-analytical phase external quality assessment programs, laboratories have the opportunity to access to documented and objective tools to achieve continuous process improvement through interlaboratory comparison (4,8). In the last 20 years, clinical laboratory societies worldwide have developed external quality assessment programs for the pre-analytical phase based on different types of strategies, namely: i) procedure recording (Type I-strategy applied in Spain, Norway, Germany, Finland); ii) distribution of simulating error samples (Type II-strategy applied in Denmark, United Kingdom, Switzerland, Sweden, Italy, Austria, Luxembourg, France, the Netherlands, Spain); and iii) QIs recording (Type III-strategy applied in the United States, Australia, Spain, Norway, United Kingdom, Brazil) (1,4,9). Among the programs that apply Type III-strategies is the project “Laboratory Errors and Patient Safety” (WG-LEPS) of the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) (10). The WG-LEPS developed a harmonized model of QIs for the clinical laboratory TTP as well as a project to record them through an online platform within which laboratories from all over the world can record their measurements and evaluate their performance by interlaboratory comparison (8,10). In Argentina, the Pre-Analytical Phase External Quality Assessment Subprogram (preEQA Subprogram) was created in 2016 within the framework of the External Quality Assessment Program (EQA Program) of the Fundación Bioquímica Argentina (11) with the purpose of contributing to the continuous improvement of the clinical laboratory TTP. The specific aims of the preEQA Subprogram are to provide professional updating on the pre-analytical phase and to carry out interlaboratory comparison of procedures (Type I-strategy) and QIs (Type III-strategy) taking into account the context and characteristics of the participating laboratories (4,9). Based on the above, the purpose of the present work was to report the results collected from the first interlaboratory comparison of four QIs carried out by the preEQA Subprogram in Argentina during 2021 and 2022.

Materials and methods

Characteristics of the preEQA Subprogram

To be eligible for participation in the preEQA Subprogram, laboratories should be registered in the EQA Program and should apply for registration in the preEQA Subprogram. The latter consists of four annual surveys conducted through the Fundación Bioquímica Argentina web page (11). As both participation in the preEQA Subprogram and survey response submission are voluntary, the number of participants and responses received varies with time. After each survey, the participating laboratories receive a report of the results collected together with a commentary on the analysis and interpretation of the results, with recommendations and related literature.

QIs evaluated in the preEQA Subprogram

The preEQA Subprogram proposed to the participating laboratories to measure four QIs. Instructions on the registration of each QI were first delivered to the participating laboratories and the measurement of these QI was subsequently carried out following the survey schedule designed by the EQA Program for the years 2021 and 2022. The QIs evaluated were designed based on the IFCC QI model (10) with self-adaptations to facilitate their measurement taking into account previously observed characteristics of the laboratories participating in the preEQA Subprogram (12). This is the reason why the four QIs were measured only in the outpatient setting to standardize the interlaboratory measurement and with a bimonthly periodicity so that the small laboratories could obtain a significant number of records.

Table 1 lists the QIs evaluated chronologically.

Table 1:

Quality indicators evaluated in the preEQA Subprogram

Quality Indicator (QI)	Data Collection Period	Formula %
NS-QI: percentage of patients recalled for a new blood sample collection due to pre-analytical causes	May-June/2021	= 100 x (number of patients recalled for a new blood sample collection due to pre-analytical causes / total number of patients of the laboratory)
CS-QI: percentage of clotted samples	October-November/2021	= 100 x (number of clotted samples from hemogram and coagulation tests / total number of samples from hemogram and coagulation tests)
HS-QI: percentage of hemolyzed samples	April-May/2022	= 100 x (number of hemolyzed clinical chemistry samples / total number of clinical chemistry samples)
TE-QI: percentage of requests with transcription errors entered into the LIS	October-November/2022	= 100 x (number of requests with transcription errors entered into the LIS / total number of requests entered into the LIS)

preEQA Subprogram: Pre-Analytical Phase External Quality Assessment Subprogram; QI: quality indicator; LIS: laboratory information system

Pareto Chart. Abscissa: pre-analytical causes predefined by the preEQA Subprogram for which patients were recalled for a new blood sample collection. Bars and left axis: number of laboratories that reported each pre-analytical cause for recalling patients for a new blood sample collection. Continuous line and right axis: accumulated percentage. Dotted lines: causes of patient recalling which, based on Pareto principle, have highest impact on the laboratories that responded to the survey. NS-QI: percentage of patients recalled for a new blood sample collection due to pre-analytical causes; B: blood sample; U: urine sample.

The percentage of patients recalled for a new blood sample collection due to pre-analytical causes (NS-QI) was designed for a first assessment of the pre-analytical phase among the laboratories participating in the preEQA Subprogram. The participating laboratories were asked to record the NS-QI as well as the pre-analytical causes due to which patients were recalled. A predefined list of pre-analytical causes, which are shown in Figure 1, was distributed to all the participating laboratories in order to standardize this registry. In addition, the laboratories were asked to group the pre-analytical causes that were not included in this predefined list under the category “other causes”. The NS-QI was adapted for the present study from that proposed by the IFCC QI model. According to the latter, the number of patients recalled for a new blood sample collection as a result of any type of error should be measured as an indicator of the outcome of the clinical laboratory TTP (10,12). In the present study, the pre-analytical causes for which patients were recalled for a new blood sample collection were analyzed using Pareto´s principle in an Excel chart (Microsoft Office). This principle focuses efforts on either the causes or the factors that impact most on a given process by stratifying them by frequency and by considering that 80% of problems stem from 20% of causes (12,13). Based on this analysis, two QIs were selected in principle to evaluate the quality of the sample collection process, namely the percentage of clotted samples (CS-QI) and the percentage of hemolyzed samples (HS-QI) (5). As for the clotted samples, the IFCC QI model proposes to measure the CS-QI on either all the samples or tubes with anticoagulant that are checked for the presence of clots (10). In contrast, the preEQA Subprogram ordered to consider only all hemogram samples (EDTA tubes) and all coagulation test samples (citrate tubes) to facilitate the recording of this QI to those laboratories that are beginners in this process. In the case of the HS-QI, all laboratories, including those with an automated hemolysis index, were asked to use the same color scale (14) - which had been provided with the instructions by the preEQA Subprogram - to detect the presence of hemolysis by visual inspection and thus standardize its recording. In the present study, a sample was considered to be hemolyzed when its color was equal to or higher either than that of tube 2 of this scale or than 0.5 g/L of free hemoglobin (10,14). The IFCC QIs model determines that 0.5 g/L of free hemoglobin is the cut-off point for visual inspection and proposes to measure HS-QI on all samples that are checked for hemolysis (10). Therefore, in order to help the participating laboratories to record this QI, the preEQA Subprogram asked to use clinical chemistry plasma samples (tubes with heparin) and clinical chemistry serum samples. The percentage of requests with transcription errors entered into the laboratory information system (TE-QI) was subsequently evaluated in order to measure the quality of the administrative process (5). On account of the fact that in Argentina the majority of laboratories receive requests written in handwriting on paper, for this QI the participating laboratories were asked to consider requests entered into their information system with any type of transcription error (e.g., in patient or physician data, in omitted, wrong or added tests, etc.) detected outside the administrative entry procedure (i.e., detected at the moment of blood collection, sample processing, results validation, report delivery, in reply to the patient or physician´s claim, etc.). In contrast, in the IFCC QIs model, only test transcription errors are considered (10). Apart from the instructions, the participating laboratories were also provided with an Excel worksheet (Microsoft Office) which was designed to record each QI and included a formula for the percentage calculation of the QIs and the instructions for obtaining the short term Sigma metrics with the calculator available at www.westgard.com (15,16). Every laboratory uploaded its results through Fundación Bioquímica Argentina web page (11) within the established times. Those responses that were implausible (e.g., percentages higher than 100%) were eliminated from the data analysis (12). Percentiles 25 (p25) and 75 (p75) of the percentage value distribution were obtained using the software Statistical Package for Social Science 15.0 (Chicago, IL, USA). Laboratory performance was classified into three levels, namely i) high, i.e. with percentage values ≤ p25; ii) medium, i.e. with percentage values between p25 and p75; and iii) low, i.e. with percentage values ≥ p75) (4,8,17). The Sigma metrics corresponding to the percentage values of p25 and p75 of each QI measured was calculated using the above-mentioned calculator (15,16) in order to guarantee a universal and objective QI assessment. The Sigma metrics relates the defect error rate per million opportunities with the efficiency of the process. Based on this, the following quality levels were considered: i) minimum acceptable quality level equal to Sigma value of 3.0, which corresponds to 6.680% of error and 93.3% of yield; and ii) minimum desirable quality level equal to Sigma value of 4.0, which corresponds to 0.621% of error and 99.4% of yield (8,12,16,18,19). Where possible, the results collected in the present study were compared with those published by the IFCC QIs project in 2023. All the databases used were anonymized for the present study in compliance with the ethical requirements for data privacy and confidentiality (20).

Figure 1:

Pre-analytical causes for which patients were recalled for a new blood sample collection and number of laboratories that reported each of these causes for patient recalling during NS-QI measurement

Results

The results from the surveys carried out in the preEQA Subprogram show that 64% of the laboratories that responded to the surveys belonged to Buenos Aires province, 8% to the Autonomous City of Buenos Aires and 6% to Santa Fe province. They also show that the percentage corresponding to the laboratories from the remaining Argentine provinces was lower than those above-mentioned. Many of the participating laboratories (70%) belonged to the private outpatient setting. Approximately 5% of them attended less than 100 patients per month whereas another 5% attended more than 3800 patients per month. 80% of the laboratories reported that, prior to the preEQA Subprogram, they had not performed any external quality control of the pre-analytical phase. As for the pre-analytical procedures that are of interest to the present study, it was observed that i) 96% of the laboratories received medical requests written in handwriting on paper; ii) 80% used syringe and needle (open system) for blood collection; and iii) 90% detected the presence of hemolysis by visual inspection. Only 40% of the laboratories did report that they systematically recorded pre-analytical errors, the main reasons for not recording them systematically being not knowing how to do it, not having time to do so and having the belief that the number of pre-analytical errors made is not enough to justify recording them.

Table 2 shows the number of laboratories participating in the preEQA Subprogram that measured four QIs and the number of valid responses received for each of them.

Table 2:

Number of laboratories participating in the preEQA Subprogram for the measurement of four QIs and number of valid responses per QI

	Percentage of the laboratories that participated within the scheme of the preEQA Subprogram to measure QIs (received responses/total participants)	Percentage of valid responses (valid responses/received responses)
NS-QI	26% (113/406)	73% (83/113)
CS-QI	56% (226/407)	100% (226/226)
HS-QI	65% (262/405)	94% (246/262)
ET-QI	52% (240/427)	93% (222/240)

QI: quality indicator; preEQA Subprogram: Pre-Analytical Phase External Quality Assessment Subprogram; NS-QI: percentage of patients recalled for a new blood sample collection due to pre-analytical causes; CS-QI: percentage of clotted samples; HS-QI: percentage of hemolyzed samples; ET-QI: percentage of requests with transcription errors entered into the laboratory information system. A response was considered to be invalid when the value reported was higher than 100%.

Figure 1 shows the number of laboratories that detected the pre-analytical causes for which patients had been recalled for a new blood sample collection during the recording of NS-QI and Pareto principle-based analysis (12,13). Table 3 lists the results collected for each QI in the present study and those reported as quality specifications by IFCC QI project (8).

Table 3:

Results obtained for four QIs evaluated in the preEQA Subprogram and their comparison with those reported as quality specifications by the IFCC QI project (8)

	preEQA Subprogram PERFORMANCE LEVEL		IFCC PERFORMANCE LEVEL (8)*
	HIGH	LOW	HIGH	LOW
	≤ p25% (p25 CI-95%)% [Sigma]	≥ p75% (p75 CI-95%)% [Sigma]	≤ p25%* (p25 CI-95%)%* [Sigma]	≥ p75%* (p75 CI-95%)%* [Sigma]
NS-QI	≤ 0.180% (0.110-0.340)% [4.5: desirable]	≥ 1.640% (1.210-2.220)% [3.7: acceptable]	Unpublished Data	Unpublished Data
CS-QI	≤ 0.000% (0.000-0.000)% [6.0: desirable]	≥ 0.245% (0.200-0.395)% [4.4: desirable]	≤ 0.126% (0.100-0.150)% [4.6: desirable]	≥ 0.527% (0.407-0.630)% [4.1: desirable]
HS-QI	≤ 0.330% (0.265-0.455)% [4.3: desirable]	≥ 2.578% (1.840-3.239)% [3.5: acceptable]	≤ 0.456% (0.000-0.739)% [4.2: desirable]	≥ 1.650% (1.590-1.820)% [3.7: acceptable]
TE-QI	≤ 0.558% (0.400-0.710)% [4.1: desirable]	≥ 5.702% (4.440-6.660)% [3.1: acceptable]	≤ 0.117% (0.078-1.105)% [4.6: desirable]	≥ 2.217% (1.705-2.518)% [3.6: acceptable]

The medium performance level is defined by the range p25-p75.

*Performance level considered as quality specifications in 2023(8). preEQA Subprogram: Pre-Analytical Phase External Quality Control Subprogram; IFCC: International Federation of Clinical Chemistry and Laboratory Medicine; NS-QI: percentage of patients recalled for a new blood sample collection due to pre-analytical causes; CS-QI: percentage of clotted samples; HS-QI: percentage of hemolyzed samples; TE-QI: percentage of requests with transcription errors entered into the laboratory information system; p: percentile; CI-95%: 95% confidence interval.

Discussion

Laboratories showed interest in participating in the interlaboratory comparison of the QIs proposed by the preEQA Subprogram. In this study, it was observed that the percentage of laboratories that measured the last 3 QIs (i.e., CS-QI, HS-QI and TE-QI) duplicated with respect to that of the first QI (NS-QI) measured. In the IFCC QI project, the number of laboratories that measured each pre-analytical QI during 2021 ranged from 25 to 289(8); in the Spanish Preanalytical Quality Monitoring Program (SEQC), 72 laboratories participated during 2018-2019(4); and in the Programa de Benchmarking e Indicadores Laboratoriais from Brazil, the number of responses obtained during 2016-2018 ranged from 34 to 1081 depending on the QI measured (12). Thus, the fact that between 83 and 262 laboratories responded to preEQA Subprogram surveys is indeed extremely encouraging particularly if one takes into account the data collection period, which corresponded to the SARS-CoV-2 pandemic context, mainly the year 2021. A plausible explanation for this - although there is still no published evidence - seems to be the fact that in Argentina the number of small and medium-sized laboratories is larger than that of large laboratories in contrast to other countries where centralized laboratories predominate and serve a large number of patients. The fact that it was nonetheless observed that a high percentage of the laboratories enrolled in the preEQA Subprogram did not measure any Qis, highlights the need to implement measures to determine the reasons for such non-participation. As regards the number of invalid responses received in the preEQA Subprogram, it was observed that the highest percentage of invalid responses corresponded to the NS-QI, which was the first QI measured by the participating laboratories. In contrast, the percentage of invalid responses for the remaining QIs was lower probably because the participating laboratories were already familiarized with the Excel chart and the web page to upload the results collected. Measuring the NS-QI proved to be useful to select future QIs in terms of their degree of priority, particularly those related to sample quality. The Pareto chart showed that, among the 10 pre-analytical causes for which patients were recalled for new blood sample collection and which impacted most on the group of laboratories that responded to the survey, clotted, hemolyzed and insufficient blood samples represented an accumulated frequency of 40%, followed by other causes that will be further taken into account for the planning of future QI measurements. It should be noted that not properly collected urine samples were included among these other causes. This is due to the fact that, although national recommendations discourage 24h urine sample collection tests in Argentina (21), they are very commonly performed (for example, for the measurement of albuminuria and proteinuria and creatinine clearance). In the case of the CS-QI, it was observed that the percentage values recorded in the present study were lower than those reported by the IFCC QI project (8), although in both cases the Sigma metrics indicated that such values achieved a desirable level of quality. In this respect, it seems likely that the higher percentage of clotted samples detected by the laboratories participating in the IFCC QI project is due to a higher availability or use of automated pre-analytical and analytical platforms, which are more sensitive than visual inspection, which is mostly used in preEQA laboratories to detect the presence of clots (4,22-24). In the case of the HS-QI, the p75% value collected in the present study was higher than that reported by the IFCC QI project (8) for this same QI obtained by visual inspection. The difference in the results observed could be attributed to the use of the open system (syringe and needle) for blood collection among the preEQA Subprogram laboratories, instead of the closed system (vacuum) which presents a lower risk of hemolysis and is also internationally recommended (25). In parallel, as visual inspection for hemolysis detection involves a certain degree of subjectivity in the results of the preEQA Subprogram, as well as in those of the IFCC QI project, it should not be considered as the main cause of the difference observed. Finally, although 75% of the laboratories responding to the survey achieved - as is the case internationally - a quality level ranging between acceptable and desirable, the low-performing labeled laboratories were suggested to concentrate their efforts on the implementation of actions aimed at improving the blood collection procedure The percentage values collected for the TE-QI were also found to be higher than those reported by the IFCC QI project (8) and the difference among the percentage values was - in the particular case of this QI - also higher, as evidenced by the Sigma metrics. This is a plausible outcome on account of the fact that the IFCC QI model takes into account only transcription errors of the tests requested. Therefore, although it is not possible to make a straightforward comparison of the results collected, such a comparison could only be orientative. Furthermore, although 75% of the participating laboratories achieved a level of quality ranging between acceptable and desirable, it is advisable that the low-performing labeled laboratories take extreme measures to improve the data entry and their control procedure. The percentage values obtained for the four QIs evaluated allowed the preEQA Subprogram and the laboratories that measured them to compare their performance with each other and with the international literature available. This, in turn, helped them. not only to determine the level of error in their pre-analytical processes but also to predict the high or low need to implement continuous improvement actions in the processes, particularly in the case of the low-performing labeled laboratories. All in all, the Sigma metrics proved to be useful to achieve an objective interpretation of the percentage values collected after measuring QIs in the preEQA Subprogram and for each individual laboratory that performed QI measurements. The decision to determine that a Sigma value of 3.0 is equivalent to a minimum acceptable quality level should not overshadow the established goal of a desirable quality level equivalent to a Sigma value either higher than or equal to 4.0. In this sense, that a laboratory achieved a medium performance level according to the distribution of percentage values of the QI in the group (p25-p75) but with an error rate > 0.819% (Sigma < 4.0 and < 99.4% of yield) means that the process in question is objectively vulnerable and does require the implementation of improvement actions (16,19). Likewise, a low performance level but with an error rate > 8.076% (Sigma < 3.0 and < 93.3% of yield) should be interpreted as unsatisfactory low performance. Sigma metrics turns out to be a fundamental tool for a correct interpretation of the results derived from QI measurement (19).

Conclusion

The majority (75%) of the laboratories participating in the preEQA Subprogram to measure the four QIs evaluated in this study showed a minimum acceptable level of quality (Sigma > 3.0). It was also observed that both the HS-QIs and the TE-QI deserve more attention. In spite of its weaknesses, the importance of this pioneering pre-analytical phase external quality assessment conducted in Argentina lies in that the participating laboratories - some for the first time - could identify processes in need of improvement thanks to the interlaboratory comparison of their performance. This preEQA Subprogram experience has allowed us to document issues deserving improvement in this Subprogram and has, in parallel, been motivating to keep on working on the measurement of pre-analytical QIs for the continuous improvement of the clinical laboratory TTP in Argentina.