Practices and Perceived Value of Proficiency Testing in Clinical Laboratories

Abstract

Background:

Proficiency testing (PT) can have regulatory and nonregulatory uses, providing an effective tool for quality improvement. Clinical laboratories were surveyed to determine how they perceive PT and how they use PT results and materials to improve laboratory testing quality.

Methods:

All laboratories certified to perform nonwaived testing under the CLIA regulations expected to perform required PT were invited to participate in the survey. We examined respondents’ use of PT from 5 laboratory types: hospital, independent, public health, physician office, and “all other.” Respondents’ awareness of resources about PT was also examined. Several questions allowed responses on a categorical scale.

Results:

Varying proportions of the respondents (n = 769) used PT to identify problems in the preanalytic (48%), analytic (86%), and postanalytic (76%) phases of testing. Responses also showed that PT was important for demonstrating personnel competency (93%), inappropriate specimen handling (80%), incorrect result interpretation (84%), and other uses. Respodents purchased PT even when not required to do so (77%). Based on all responses, most considered PT worth the cost (65%).

Conclusions:

Laboratories, regardless of type, have found ways of using leftover PT samples and the information from PT event summaries to help improve laboratory quality. Our findings suggest many laboratories are not taking full advantage of PT to improve testing quality. Additionally, the study suggests a need to improve awareness of resources about PT.

BACKGROUND

Proficiency testing (PT)³ is mandated for clinical laboratories performing nonwaived testing as part of the CLIA of 1988 (1). Before CLIA 1988, CLIA 1967 had only required successful PT performance for laboratories that referred patient specimens across state boundaries (2).

As an external check against peer laboratories or reference methods, PT is considered to be an essential component of a laboratory quality management system and can be an effective tool for quality improvement (3, 4). For example, longterm analytical performance can be tracked using information in PT summary reports, and residual PT samples may be used for other purposes (5). These nonregulatory uses may be underappreciated.

We sought to better understand how clinical laboratories in the US use PT to improve patient testing and identify possible gaps. The CDC and the Association of Public Health Laboratories (APHL) developed a national survey addressing PT in the nonmicrobiology specialties of immunology, endocrinology, routine chemistry, toxicology, and hematology. Here we summarize the results regarding PT use for analytes not specifically required by the CLIA regulations. While the results may not be generalizable because of the small and nonrepresentative sample, the data and statistics provide laboratory managers with useful information.

MATERIALS AND METHODS

The survey was reviewed and approved by the Office of Management and Budget before its release (control number 0920–0961). All CLIA-certified laboratories that perform nonwaived testing were invited to take the online survey. The survey sample was recruited using a mass-mailed trifold brochure and postcard reminders to 37216 laboratories and advertisements in various trade and professional magazines. The survey was open from July 27, 2013, until November 20, 2013. A copy of the survey is available at aphl.org/ptsurvey.

Survey responses were linked to a laboratory’s CLIA number or to a unique code linked to existing administrative CLIA data to minimize inconvenience to respondents and increase reliability of the data. Use of either option prevented redundant entries from the same laboratory, so no laboratories were overrepresented. Demographic data included reported annual test volumes, address, certificate type (compliance or accreditation), and laboratory type. Only anonymized, aggregated results were provided to the CDC and APHL. Sentient Research compared responses and CLIA self-reported laboratory type (hospital, independent, physician office laboratory, and public health). Laboratories that were not identified as any of the four CLIA laboratory types were placed in the “all other” category.

RESULTS

There were 871 online surveys completed. After removal of responses that had invalid identification numbers (68), duplicated (28), or unlinkable (6) CLIA numbers, 769 valid completed survey responses were obtained. The overall distribution of CLIA-certified laboratories (n = 37216) that perform PT compared by laboratory type was 50% physician office, 19% hospital, 19% of all other, 12% independent, and 0.6% public health (PH). The “all other” category included ancillary test sites, community clinics, and various laboratory testing sites represented 10 times or less. In contrast, the distribution of valid completed responses was 24% physician office, 46% hospital, 16% all other, 8% independent, and 6% PH. Therefore, the responses were underweighted in physician offices and over-weighted in hospitals and PH laboratories.

Alternative (nonregulatory) uses for PT

Many possible uses for PT, in any of the 3 phases of testing, were considered “important” by most participants (Fig. 1). Almost half of respondents (48%) said that they use PT samples to identify some problems in the preanalytic phase of testing. Assessment of personnel competency was the most frequently and consistently cited preanalytic use of PT and the most common response across all laboratories. Other preanalytic uses listed in the survey were common across all laboratory types but had larger response variability when looking at the 25th–75th percentile (interquartile ranges), particularly responses for “inappropriate specimen handling, including dispensing” and “inappropriate storage of samples.”

Fig. 1. Analysis of alternate uses of PT throughout all phases of testing that were considered important by respondents.

The minimum and maximum data points are shown at each end of the whiskers. The box contains 50% of the data, and the median is the line inside the box. The bottom of the box is the first quartile data point, and the top of the box is the third quartile data point.

As expected, a majority of respondents use PT to identify issues during the analytic phase (86%). The most common of these was using PT to identify instrument problems. The interquartile ranges for those who used PT for identifying calibration errors and specimen dilution errors are almost identical across laboratory types; however, the largest range was seen for identifying errors when extracting an analyte from the sample. The majority of PH laboratory respondents (68%) use PT for this reason, while few respondents from the “all other” laboratory category (28%) do.

Most respondents (67%) indicated they use PT to identify problems in the postanalytic phase. Greater than 50% of respondents across all laboratory types use PT in this phase to identify transcription errors, incorrect test result interpretation, incorrect calculations, and delayed reporting to PT programs. The widest range of responses based on laboratory type occurred with the identifying incorrect calculations option. Only 66% of PH laboratories use PT for this purpose, while 83% of hospitals do.

Voluntary PT participation.

Overall, 79% of respondents said they purchase PT for analytes beyond those required by CLIA regulations (n = 696). We omitted responses for this particular question from laboratories accredited by the College of American Pathologists (CAP) because they are required to enroll in the CAP PT program for each analyte tested, if available. Of the responses from these laboratories (n = 374), 77% indicated that they purchase some nonrequired PT; 39% for 1–5 analytes, 17% for 6–10 analytes, 10% for 11–20 analytes, and 10% for more than 20 analytes. Reasons included competency assessment (73%), accuracy tracking (62%), accreditation organization requirements (62%), identification of problems (62%), instrument performance assessment (59%), continuing education (51%), and meeting CLIA’s requirement for biannual verification of accuracy of nonregulated analytes (43%).

Other PT uses considered beneficial.

The most common alternate use of PT considered “very beneficial” was the opportunity to identify educational areas needing improvement. Those uses that had the lowest percentage of responses but the smallest range across laboratory types were making recommendations for methodology or instrument changes, troubleshooting assays, and comparing instruments before purchase (Fig. 2). Despite the utility of PT, more than 10% of respondents from all laboratory types except PH laboratories (8%) considered PT “somewhat more costly than its value.” Most respondents (65%) said that PT is “clearly” or “somewhat” worth the costs.

Fig. 2. Analysis of benefits of PT considered beneficial by the respondents.

The minimum and maximum data points are shown at each end of the whiskers. The box contains 50% of the data, and the median is the line inside the box. The bottom of the box is the first quartile data point, and the top of the box is the third quartile data point.

Awareness of resources

We asked respondents to rate several sources of PT information. Overall, respondents were most aware of CLIA PT brochures available from Centers for Medicare and Medicaid (CMS) (86%) CLSI GP27, “Using Proficiency Testing to Improve the Clinical Laboratory” (82%); CLSI 29, “Assessment of Laboratory Tests Where Proficiency Testing Is Not Available” (81%); and CDC publications and online resources (73%). Responses of “not aware of resource” occurred <10% for most sources. Hospital and independent laboratories were the most aware of CLSI guidelines. PH laboratories were most aware of CDC resources.

DISCUSSION

Leveraging PT to improve quality in nonregulatory ways makes sense, since laboratories already have the samples and summary information from each PT event. This study sought to understand how PT is perceived and used beyond regulatory requirements. Some reported using PT for CLIA-required biannual accuracy checks of nonregulated analytes. Although most PT materials are not accuracy based, this is valid because accuracy can be compared externally to the peer group summary results and tracked. A minority of respondents (15%) answered incorrectly that PT is potentially important or important to identify sending tests to the wrong reference laboratory for confirmatory testing. This scenario is considered PT referral and is disallowed by CLIA. Some responses may reflect the understanding that PT referrals are improper; some respondents may be confused on this issue. Regardless, all clinical laboratory employees, including “send-out” staff, should be informed that PT referral is never allowed. Other PT uses reported were to assess instrument performance, competency of testing personnel, and continuing education needs; introduce new assays; and track accuracy of methodology. Yet, about 10% of respondents still perceive PT as more costly than its value. Still, there are opportunities to promote increased use of PT such as using PT event summary data when considering new testing equipment. Additionally, the finding that many laboratories not accredited by CAP purchase PT for analytes not required by CLIA appears to support the idea that laboratories derive multiple benefits beyond meeting regulatory requirements.

Lack of awareness of multiple sources of information about PT (CLSI, CDC, and CMS) was expected. Based on the responses, we believe there is a great opportunity to educate laboratories and suggest more training about all resources available.

Several caveats are associated with this study. The limited response rate and the relative overrepresentation of responses from hospital, independent, and PH laboratories require cautious interpretation of our findings. These laboratories, and especially the subset of hospital laboratories that hold a Certificate of Accreditation from CAP, the Joint Commission, or other accreditation organizations, may be more likely to view PT favorably. Therefore, our findings may misrepresent the usage of PT for nonregulatory purposes and underrepresent the opportunities to increase its usage. This bias may have been caused by (a) anxiety over regulatory and financial consequences, (b) misdirected brochures, (c) internal laboratory policies preventing commenting on their practices, (d) the length of the survey, or (e) a perception that laboratories could have been identified. Despite these limitations, this report should provide more awareness of how PT can be useful to improve quality of laboratory testing and of the resources about PT that are available.

IMPACT STATEMENT.

Clinical laboratory personnel may underappreciate the various ways proficiency testing (PT) can be used to improve laboratory quality. Alternative uses for PT in clinical and public health laboratories, beyond minimally fulfilling regulatory obligations, are presented and discussed. The information originated from a survey given to clinical laboratory personnel across the United States.