Risk Management for Point-of-Care Testing

Abstract

Point-of-care testing (POCT) is growing in popularity, and with this growth comes an increased chance of errors. Risk management is a way to reduce errors. Originally developed for the manufacturing industry, risk management principles have application for improving the quality of test results in the clinical laboratory. The Clinical and Laboratory Standards Institute (CLSI), EP23-A Laboratory Quality Control based on Risk Management guideline, introduces risk management to the clinical laboratory and describes how to build and implement a quality control plan for a laboratory test. A simple, unit-use blood gas analyzer is utilized as an example for developing a laboratory quality control plan. The US Centers for Medicare and Medicaid Services (CMS) has revised the Clinical and Laboratory Improvement Amendments (CLIA) interpretive guidelines to provide a new quality control option, individualized quality control plans (IQCP), for decreasing the frequency of analyzing liquid controls from two levels each day of testing to manufacturer recommended frequencies in conjunction with a device’s built-in internal control processes and the risk of error when testing with that device. IQCPs have the advantage of allowing laboratories the flexibility to adopt alternative control processes in concert with traditional liquid controls to improve efficiency and cost effectiveness while providing optimal quality POCT results for patient care.

INTRODUCTION

Point-of-care testing (POCT) is an increasingly popular means of delivering laboratory tests close to the patient. POCT allows for rapid diagnostics and turnaround of test results to provide for faster medical decision-making and improved patient outcomes (1). However, if inappropriate samples are collected, specimen is mislabeled, analysis is performed incorrectly, or test is misinterpreted, wrong results may be reported and acted on by the clinician. Studies have indicated that for central laboratory testing, most errors occur in the preanalytical phase, prior to the sample arriving in the lab (2). For POCT, the majority of errors occur in the analytic phase of testing (3). In fact, errors can occur in any phase of laboratory testing whether performed in a central laboratory or at the point-of-care. As laboratory directors, we should know our processes and take steps to detect and prevent errors before those mistakes reach the clinician and affect patient care.

Risk management is a way to reduce errors with POCT. Risk management is defined as the systematic application of management policies, procedures, and practices to the tasks of analyzing, evaluating, controlling, and monitoring risk (4). Risk is the chance of suffering harm or loss. Risk is generally assumed from the patient’s perspective, but risk can also apply to the operator of the POCT device, the laboratory administration and even the hospital and its reputation. Risk is the chance of suffering harm or loss, and risk can be estimated through a combination of the probability of occurrence of harm and the severity of that harm(5). Errors that occur more frequently have greater risk, and errors that lead to greater harm also present greater risk. So, there is a spectrum of risk from low to high. Once can never get to zero risk. There is always some chance of risk. Our job as laboratory staff is to maintain risk to a clinically acceptable level. Risk management should not be a new concept. Laboratories conduct a number of activities to limit their chance of errors. The performance of new tests is validated before use on patients. Staff troubleshoots control failures and followsup on complaints from clinicians. When errors are detected, the harm to patients is estimated and actions are taken to prevent recurrence in the future. So, risk management is simply a formal term for many of the activities that laboratories are already doing.

QUALITY CONTROL

Quality control is a means of detecting and preventing errors. Besides frequency and severity, detectability is a third factor in the risk estimation equation. Quality control and risk management principles were developed from the manufacturing industry. As products are constructed on a factory line, the quality of the product is inspected to ensure that it meets manufacturer specifications. If problems are noted, the line can be stopped and production corrected to ensure the quality of the final product.

These industrial risk management principles have application for reducing errors in laboratory testing as well. In the central laboratory, batch reagents are used for several days. During that time, the regent can drift and degrade impacting the test result. Laboratories analyze liquid quality control, a stable sample with predefined acceptability ranges, in order to detect reagent problems before they affect the test result. Traditionally, liquid controls are analyzed at two concentration levels each day of testing or more frequently as required by the stability of the test system.

Liquid quality controls do a good job at detecting systematic errors. These are errors that affect the patient sample in the same manner as the quality control sample. Reagent degradation, calibration errors, dilution and pipetting errors are examples of systematic errors that quality control can effectively detect and prevent before the errors affect a patient result. Quality control, however, does a poor job at detecting random errors which uniquely affect individual samples. Bubbles, clots, drugs, hemolysis and other sample specific errors are not detected by liquid quality control. Other mechanisms, like bubble and clot detection, or analyzer hemolysis indices must be utilized to detect random errors. So, analyzing two levels of liquid quality control each day of testing does not entirely eliminate risk, and laboratories have still produced bad results despite analyzing liquid quality control.

Newer POCT devices utilize unit-use cartridges or test kits. Analysis of liquid quality control consumes the entire test in the process, and there is no guarantee that the next test will perform identically. Alternative control processes must be used for these devices in addition to liquid controls in order to optimize the quality of these tests. Many POCT kits have built-in biologic and chemical controls to ensure the performance of individual tests. Fecal guaiac occult blood cards have a positive and negative control area on each card to ensure the reactivity of the card and developer. Urine pregnancy tests have a control line on each test to verify test storage and viability of the antibodies on the test. Drug, rapid strep, HIV and other POCT unitized tests have similar control lines or regions that guarantee the quality of the test kit and result with each test.

These control lines are control processes that act as an alternative to traditional liquid controls in order to detect the risk of specific errors when using those tests. Some tests, like bilirubinometers, cannot even accept a liquid sample, so alternative control processes must be utilized to ensure the quality of this test. Consider molecular testing where hundreds of reactions may occur on a single chip. How does a laboratory effectively control the quality of these tests? It is neither economical nor possible to analyze two levels of liquid controls for every reaction on this test each day. The effective way to ensure quality would consider risk of those errors that are most likely to occur or cause greatest severity of harm from an incorrect result. The amount and quality of specimen, the reactivity of the replicating enzyme, and the thermocycling of the device are key failure points, and those are the steps that should be monitored by the quality processes.

Laboratories must partner with the manufacturer to develop an effective quality control plan. Although the practice of analyzing two levels of liquid quality control have given laboratories some degree of assurance that results are valid, newer devices have built-in electronic controls, and on-board chemical and biological controls, No single quality control procedure can cover all devices, since devices may differ in design, technology, function, and intended use (6). Quality control information from the manufacturer increases the user’s understanding of device overall quality assurance requirements, so that informed decisions can be made regarding suitable control procedures. Manufacturers understand their devices and the limitations of those devices, while laboratories know how the device will be utilized and test results applied for patient care. A quality control plan identifies the weaknesses in the testing process and defines the roles of the manufacturer built-in control processes and laboratory actions required to maintain risk to an acceptable level.

The Clinical and Laboratory Standards Institute (CLSI) document EP23-A introduces the industrial risk management principles to the clinical laboratory (7). EP23 describes good laboratory practice for developing a quality control plan based on manufacturer’s information, applicable regulatory and accreditation requirements, and the individual healthcare and laboratory setting. This guideline recommends collecting information about a test system and processing that information through a risk assessment to develop a quality control plan. The testing process is mapped from preanalytic through analytic and postanalytic phases. Weaknesses in the testing process are identified and for each hazard identified, the laboratory defines a control process which will detect and prevent that error, controlling risk to a clinically acceptable level. Some hazards, like use of expired reagent, may be effectively controlled through a manufacturer built-in process such as barcoding which prevents the operator from utilizing expired reagents. Other hazards may require the laboratory to take an action, like instrument maintenance or operator training/competency. A quality control plan is essentially a summary of all the hazards considered and laboratory actions required to minimize risk. Once developed, the quality control plan is implemented and monitored for effectiveness. If errors continue to occur, the laboratory is encouraged to troubleshoot, reassess their risk and modify the quality control plan as required.

RISK MANAGEMENT EXAMPLE

A unit-use blood gas device may be used as an example of the risk management process. The first step to build a quality control plan is to collect information about the test. Let’s consider a generic POCT blood gas and electrolyte analyzer intended for use in a same-day surgical center. The need for testing is low, only 1 – 2 tests per day. At a cost of $10 – 20 per test, the requirement to perform two levels of liquid control each day of testing will increase the cost of testing significantly and add to the turnaround time of results since control results will need to be evaluated before patient testing can be conducted. The use of alternative control processes provided by the manufacturer will improve cost, test and labor efficiency.

Review of the package insert allows the laboratory to determine intended use, test system operation and test limitations. The system is a portable clinical analyzer for the in vitro quantification of various analytes in whole blood. The test system consists of the portable clinical analyzer, test cartridges sealed in a foil pouch for protection during storage, quality assurance materials (liquid control and calibration verification solutions), and a data management system with a server class computer, data management software, wireless connectivity, and laboratory and hospital information system interfaces. The unit-use cartridge contains all the components to perform testing including; a calibrant solution, reagents, sample handling system, and sensors. The analyzer automatically controls all steps of the testing process such as fluid movement, calibration, fluid mixing, and thermal control. The cartridges are standardized to plasma core-laboratory methods using multi-point calibration curves stored in the device memory that are stable over many lots. Upon insertion, a calibrant solution in the cartridge is passed over the sensors. Signals produced by the sensor responses to the calibrant solution are measured, and a one-point calibration adjusts the sensor offset to the stored multi-point calibration curve. The analyzer then moves the sample over the sensors and the signal of the sensor responses to the sample are measured from the adjusted calibration curve.

Examination of the manufacturer, internal control processes allows an understanding of how the process functions and what errors can be detected and prevented with that process. The blood gas and electrolyte analyzer contains simulated internal control processes that check the edge connector, internal electronics and analyte circuitry. The internal control simulates the electronic signals that are produced during a cartridge test. An isolated region of the internal circuit board sends a range of simulated sensor signals through the cartridge measurement channels. The range of signals encompasses the entire linear range expected from blood analytes. Next, conductivity out of the connector pins is measured, insuring no contamination is present on the edge connector which would interfere with the test. Signal measurements must fall within strict predetermined thresholds in order to pass. The internal simulated control is performed automatically every 8 hours or if there has been a significant change in analyzer temperature, from cold to hot, since this can cause condensation on the connectors. The internal control can also be performed manually whenever the performance of the device is in question. Internal simulated controls are never intended to entirely replace liquid quality control, and the manufacturer recommends analyzing liquid controls with each shipment of cartridges, new lots of cartridges, whenever cartridges experience a temperature shift >8°C, or as required by the laboratory. Temperature is monitored continuously during each test, but a temperature verification cartridge is recommended at least annually.

The information about the test system and the function of the internal control processes can now be processed through a risk assessment. Risk assessment is best started by mapping the testing process to look for weaknesses and steps that could lead to error. Follow the sample from order to specimen collection, analysis, and reporting of results. Areas of focus should include the sample, the reagents, the operator, the analyzer, and the environment. Examine those hazards of greatest risk first including errors that occur frequently or lead to greater severity.

For compliance with federal and state regulations, testing should only be conducted based on a physician order. With POCT, operators can simply pick up the device and perform a test. So, operators must be trained to only conduct a blood gas or electrolyte test with this system based on an existing physician order. This should become an element of the operator training program. With appropriate training and demonstration of ongoing operator competency, the laboratory can conclude that risk of this error is reduced to a clinically acceptable level.

Blood gas samples should be collected anaerobically in electrolyte balanced heparin. Inappropriate collection or use of the wrong specimen additive can affect blood gas and ionized calcium results. Operators should thus be trained to utilize the appropriate sample and collection technique. Failure to adequately mix or over-mixing the sample can further lead to clots or hemolysis of the sample. Whole blood samples continue to metabolize after collection, so prompt analysis, no more than 15 – 30 minutes after collection, is important. These are additional elements that should be added to the operator training and competency program to reduce risk of these errors.

Operator technique can impact POCT results, so the effect of operator technique is critical to assessing risk with POCT. Operator lock-out features on POCT devices require a personal identification number to unlock the device and perform patient testing. This feature ensures that only those trained and competent operators are conducting testing. Adding too much or too little sample can affect test results by flooding the cartridge or contaminating the connector pins, and insufficient sample failing to adequately contact the sensors in the cartridge. This analyzer has volume detection and will not allow overfilling or start a test until an adequate amount of sample has been added. The analyzer also automates all steps of the testing process, preventing incorrect timing, misinterpretation, or other procedural steps common for POCT. The analyzer also detects the expiration date of the cartridge through barcoding, preventing use of expired reagents. Documentation of results into the patient’s medical record presents an additional step for operators, so there is a risk of manual test results not being documented. The test system wireless connectivity and data management system ensure documentation of results without need for operator intervention or requiring additional operator actions. POCT devices can transmit nosocomial infections between patients, so cleaning and disinfection between patients is important. Training and reminders for staff on proper cleaning will effectively reduce risk of this error.

The cartridges contain the chemistry and detection sensors of the test system. Exposure of cartridges to temperatures outside of manufacturer specifications during shipping and lot-to-lot variation can affect test results. Analysis of liquid quality control upon receipt of new shipments and lots of cartridges can prove the viability of the cartridge prior to use for patient samples. However, cartridges can also degrade during storage, so temperature monitoring of storage conditions is required to ensure continued viability through the life of the cartridges. Temperature monitoring of liquid control sample storage is also important to ensure control viability. Periodic analysis of liquid quality control will further ensure cartridge and control stability. At what frequency should control samples be analyzed? The manufacturer recommends testing liquid control samples upon receipt of each shipment, with new lots of cartridges, and periodically to verify cartridge stability during storage. To determine the frequency of liquid control testing during storage, laboratories can perform side-by-side testing of daily liquid controls with internal control processes to document shelf stability for a period of several weeks. Once stability is documented for several weeks, the laboratory will have data to decrease the frequency of liquid control to every few days, and eventually weekly or monthly, depending on the life-span of cartridges after receipt.

Temperature and humidity can also affect the analyzer during analysis. The analyzer automatically detects environmental conditions which will impact analysis and warn the operator. The analyzer does not require water, works on battery power and internally detects the electrical circuitry and sensor connector pins. So, these risks are not a consideration with this device.

Once the testing process has been mapped, hazards recognized and control processes identified, the third step of the risk management process is summarizing the quality control plan. The quality control plan summarizes all of the hazards recognized during the risk assessment and the error mitigations selected, both those internal control processes from the manufacturer and the actions from the laboratory. The laboratory assesses whether the mitigations reduce risk to a clinically acceptable level. If risk is not reduced to an acceptable level, then the laboratory must take additional mitigation steps to control the risk. Such actions may include additional controls, maintenance, training or other actions.

The final step of the risk management process is implementing the quality control plan and monitoring the effectiveness of the plan. Benchmarking of the laboratory’s quality can prove the effectiveness of the plan. Benchmarks for this blood gas and electrolyte analyzer could include trends in quality control, internal controls as well as liquid quality controls, analyzer error codes, physician complaints, or any other unexpected trends. When errors do arise, the laboratory should troubleshoot to determine the source, correct the process and reassess risk in light of the new information, modifying the quality control plan as required. This creates a continuous quality control process for the laboratory and this device.

CONCLUSIONS

The US Centers for Medicare and Medicaid Services (CMS) recently implemented new Clinical and Laboratory Improvement Amendments interpretive guidelines in January 2014 (8). Risk management principles have been incorporated into the new interpretive guidelines in the form of Individualized Quality Control Plans (IQCP). CMS will begin inspecting for laboratory IQCPs beginning in 2016. At that time, laboratories will have two quality control options: 1) perform two levels of liquid quality control each day of testing or 2) develop an IQCP in order to reduce the frequency of liquid quality control. The laboratory cannot reduce frequency below manufacturer recommendations, and the laboratory must perform liquid quality control at some frequency (i.e., performing no liquid quality control is not an option.). Although IQCP will initially only apply to CLIA moderate complexity devices, any laboratory will benefit from mapping their processes and assessing weaknesses in their tests.

An IQCP provides several benefits for laboratories. Since the chemistry of the test reaction is in the unit-use test cartridge, facilities with dozens of the same device can select a subset of devices and rotate the analysis of liquid controls, since all devices share the same lot and supply of unit-use cartridges. For laboratory-developed tests, the laboratory can optimize the balance of liquid controls with manufacturer internal control processes. Most importantly, by developing an IQCP the laboratory will embrace industrial risk management principles and learn how to better detect and control risks with their test systems.

Table 1.

Example Risk Assessment: Blood Gas and Electrolyte POCT Analyzer

Example risk assessment for a generic unit-use POCT blood gas and electrolyte analyzer considering risks from samples, operator, reagents, device and the environment on the testing process

Hazard	Manufacturer Control Process	Laboratory Action	Risk Clinically Acceptable?
Physician order		Operator training	Yes
Anaerobic collection for blood gases		Operator training	Yes
Incorrect tube additive		Operator training	Yes
Clots, hemolysis (undermixing or over-mixing)	Clot and bubble detection	Operator training	Yes
Delays in analysis		Operator training	Yes
Operators trained/competent	Operator lock-out		Yes
Over-filling or under-filling	Sample detection		Yes
Incorrect operator procedure	Automated test analysis		Yes
Use of expired reagents	Expiration date bar-coded in cartridge		Yes
Failure to document results	Wireless connectivity		Yes
Forgetting to clean device		Operator training	Yes
Exposure during cartridge shipment		Analyze liquid quality controls	Yes
Lot-to-lot variability		Analyze liquid quality controls	Yes
Cartridge degradation during storage		Monitor storage conditions Analyze liquid quality controls	Yes
Device failure – electrical, sensor, computational	Internal checks and internal QC	Monitor error codes	Yes
Environment temperature and humidity	Continuously monitored		Yes