Strengthening Data Defensibility in Government Human and Animal Food Testing Laboratories Through Accreditation: Lessons Learned From the FDA Accreditation Support Program

For food safety agencies, the ability to quickly remove an adulterated product from commerce is essential to protecting public health. Often, such removal follows the detection of a contaminant (eg, microbiologic, chemical, radiologic) in a product through rigorous and timely laboratory testing in a government human or animal food testing laboratory. State or federal regulatory agencies, such as the US Food and Drug Administration (FDA), the Centers for Disease Control and Prevention (CDC), the US Department of Agriculture (USDA), and state food safety regulatory programs, rely on data from such testing to support enforcement actions (eg, recalls). A critical component in establishing defensibility of these data is laboratory accreditation. Obtaining International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 17025 accreditation—an internationally recognized standard of competency to which laboratories are encouraged to adhere to be technically competent¹—requires a thorough review of a laboratory’s quality management system by an accrediting body that is a full member and signatory of the International Laboratory Accreditation Cooperation, an international organization of various accreditation bodies, including those responsible for ISO/IEC 17025 accreditation (https://ilac.org). The accrediting body reviews the laboratory’s procedures and records, and granting accreditation certifies that the laboratory is competent to perform specific tests that support regulatory decisions about food safety. Regulatory agencies are confident that the data produced by an accredited laboratory are defensible (if challenged in court) and reliable (ie, consistent across repeated measurements), facilitating a rapid response to a contaminated product.

Although accreditation is critical to determining the reliability and defensibility of laboratory data, few articles detail the efforts of the assistance programs—such as those federally funded through the cooperative agreement mechanism—that support such accreditation and their effect on food testing laboratories. Here, we describe the accomplishments of the FDA’s cooperative agreements and the partnerships among various stakeholders to support ISO/IEC 17025 accreditation of government human or animal food testing laboratories. We also discuss lessons learned and implications for the food safety community.

In 2011, Congress increased the ability of the FDA to protect the nation’s food supply through passage of the Food Safety Modernization Act.² The Act shifts the focus from responding to food contamination to preventing it through enhanced partnerships and integration among federal (FDA, CDC, USDA) and state (eg, state regulatory programs and food testing laboratories) food safety stakeholders. Accreditation instills confidence in laboratory data, which facilitates this integration through faster regulatory action. The FDA therefore sought to assist state human and animal food testing laboratories to become accredited or enhance their scope of accreditation by adding more methods or technologies through 5-year cooperative agreements.³ A cooperative agreement is a federal funding mechanism that allows the government to be substantially involved in the funded activity.

In 2012, the FDA enrolled 31 state human and animal food testing laboratories in these cooperative agreements. The eligibility was limited to the laboratories serving those state-manufactured food regulatory programs (ie, state-level regulatory agencies responsible for the oversight of food manufacturing plants) that were already enrolled in the FDA’s Manufactured Food Regulatory Program Standards (MFRPS) cooperative agreement.⁴ The MFRPS is an FDA cooperative agreement program that strives to standardize the design and management of state manufactured food regulatory programs. Both the laboratory-focused and regulatory program–focused cooperative agreements required the participating laboratories and their associated state regulatory programs to develop joint sampling plans for the scheduled collection of manufactured food products. Laboratories that sought initial accreditation were awarded $300 000 per year for 5 years, and laboratories that were expanding their accreditation scope received $150 000 per year for 5 years. The FDA has since awarded similar accreditation funding to an additional 15 state laboratories in subsequent cooperative agreement programs (Box 1). These funds were used to support activities such as hiring staff (eg, quality managers), performing gap analyses of the laboratory’s current quality management system versus what the ISO/IEC 17025 requires, and purchasing equipment and software (eg, temperature monitoring systems, laboratory information management systems, and document control software).^5,6 In addition to these cooperative agreements, in 2012, the FDA funded⁷ the Association of Public Health Laboratories (APHL), the Association of Food and Drug Officials (AFDO), and the Association of American Feed Control Officials (AAFCO)—all member-based professional associations focused on food safety–related aspects of public health—to provide technical support, training, and other resources to state human and animal food testing laboratories that were seeking or enhancing accreditation, as described hereinafter.

Box 1.

Human and animal food testing laboratories involved in FDA Cooperative Agreement Programs to achieve or expand the scope of ISO/IEC 17025^a accreditation or receiving assistance from APHL’s accreditation consultant toward achieving or expanding the scope of accreditation, United States, January 2018

Agencies With Laboratories That Were Involved in FDA Cooperative Agreement Programs

Agencies With Laboratories That Received Assistance From APHL’s Accreditation Consultant

Alabama Department of Public Healthb Alaska Department of Environmental Conservationb California Department of Food and Agricultureb California Department of Public Healthb Colorado Department of Agriculturec Colorado Department of Public Health and Environmentb Connecticut Agricultural Experiment Stationb Connecticut Department of Public Healthb Florida Department of Agriculture and Consumer Servicesc Georgia Department of Agricultureb Illinois Department of Agricultureb Illinois Department of Public Healthb Indiana State Department of Healthb Iowa Department of Agriculture and Land Stewardshipb Kansas Department of Agricultureb Kentucky Cabinet for Health and Family Servicesb Louisiana Department of Agriculture and Forestryb Louisiana Office of Public Healthb Maryland Department of Health and Mental Hygieneb Massachusetts Department of Public Healthb Michigan Department of Agriculture and Rural Developmentc Minnesota Department of Agriculturec Mississippi University, State Chemical Laboratoryb Missouri Department of Agricultureb Missouri Department of Health and Senior Servicesb Nebraska Department of Agricultureb New Jersey Department of Agricultureb New Mexico Department of Agricultureb New York State Department of Agriculture and Marketsc North Carolina Department of Agriculture and Consumer Servicesc Office of the Texas State Chemistc Ohio Department of Agriculturec Oregon Department of Agricultureb Pennsylvania Department of Agricultureb Rhode Island Department of Healthb South Carolina Department of Agricultureb South Carolina Department of Health and Environmental Controlb Tennessee Department of Agricultureb Texas Department of State Health Servicesb University of Iowab University of Nevadab Vermont Department of Health Laboratoryb Virginia Division of Consolidated Laboratory Servicesb Washington State Department of Agriculturec West Virginia Department of Agricultureb Wisconsin Department of Agriculture, Trade and Consumer Protectionc Wyoming Department of Agricultureb

Hawaii State Department of Health Michigan Department of Health and Human Services New Hampshire Division of Public Health Services New Mexico Department of Health New Mexico State University, Center for Animal Health, Food Safety and Biosecurity Oklahoma Department of Agriculture, Food and Forestry Pennsylvania Department of Health South Dakota State University Tennessee Department of Health Washington State Department of Health

Abbreviations: APHL, Association of Public Health Laboratories; FDA, US Food and Drug Administration; ISO/IEC, International Organization for Standardization/International Electrotechnical Commission.

^a General requirements for the competence of testing and calibration laboratories.

^b Laboratory was funded to achieve initial ISO/IEC 17025 accreditation.

^c Laboratory was ISO/IEC 17025 accredited before the FDA ISO cooperative agreement and was funded to expand accreditation scope.

Progress Toward Accreditation Goals

All 46 laboratories involved in the cooperative agreement as of 2018 were working toward their accreditation goal, and all but 2 of the 31 initial awardees from 2012 had achieved or expanded the scope of their accreditation by August 2017; 3 of 15 laboratories funded under subsequent cooperative agreements have already become accredited. These accredited laboratories reported that they were assisted in achieving these goals by strong partnerships among human and animal food testing laboratories (state to state, state to federal), between state regulators and their state laboratory counterparts, and between laboratories and their professional associations (APHL, AAFCO, AFDO).

Laboratory personnel reported that quality managers and technical staff members in food testing laboratories across the country were assisting one another in resolving daily challenges and sharing best practices. Regulatory food program staff members were collaborating with their laboratory counterparts on sampling plans, operating procedures, and other common issues. The participants felt that these partnerships were key to strengthening an Integrated Food Safety System,⁸ an FDA initiative to work with its state partners to reduce duplication of effort and promote data exchange. As part of these partnership-building efforts, APHL, AAFCO, and AFDO have focused their efforts on working with members and federal partners to develop resources, publish best practices, and hold meetings and trainings for laboratories to share experiences and knowledge as they work toward accreditation goals.

Professional Association Resources That Support Laboratory Accreditation

APHL hosts a curated, online, public accreditation resources web page⁹ that features laboratory accreditation success stories, quality improvement tools, and more. A restricted-access resource repository includes more than 280 documents (eg, standard operating procedures, templates, sample worksheets, quality manuals) that governmental laboratories can adopt or adapt to fit their needs. APHL also hosts an accreditation training resources web page¹⁰ that lists ISO training providers, proficiency testing providers, and accreditation-related webinars.

Working with members and federal partners, APHL developed a Quality Management Series of training materials¹¹ that provide timely and consistent quality management training tools for new laboratory hires and refresher training for veteran staff members. The materials primarily consist of customizable, online modules on topics such as traceability (the ability to discover where and how a product was made), proficiency testing, and management review.

APHL also offers direct technical assistance to 10 human and animal food testing laboratories in 9 states (2 in New Mexico) that are interested in pursuing accreditation to the ISO/IEC 17025 standard but are not participating in the FDA cooperative agreement programs (Figure). These laboratories were ineligible for the cooperative agreement because their state programs were not enrolled in the MFRPS cooperative agreement; regardless, they have a desire to become accredited.

Figure.

States with human and animal food testing laboratories involved in the US Food and Drug Administration (FDA) International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 17025 cooperative agreement.

As part of such assistance, APHL’s accreditation consultant performs laboratory gap analyses, reviews documents and quality manuals, and provides guidance throughout the accreditation process at no cost to the laboratory. Since 2012, 5 laboratories using the accreditation consultant have achieved ISO/IEC 17025 accreditation; 2 more are expected to be accredited by late 2019. Accrediting bodies require that laboratories conduct proficiency testing in the methods and matrices (or the sample components other than the analyte of interest) covered in their accreditation scope to demonstrate their ability to produce accurate data. The proficiency tests should come from a provider that is accredited to the ISO/IEC 17043 standard, an ISO/IEC standard that specifies the requirements for competence of the providers of proficiency testing schemes. One such provider, AAFCO’s Proficiency Testing Program,¹² sought to expand its test offerings and bring its Proficiency Testing Program into compliance with ISO/IEC 17043 to address the needs of state animal food testing laboratories. The expansion included adding proficiency testing schemes for pet food ingredients, minerals of toxicologic concern, and mycotoxin contaminants. The scope of the program’s Animal Food Proficiency Testing Scheme was also expanded to begin incorporating as testing items the various veterinary drugs at residue (or contamination) concentrations in a minimum of 4 testing rounds annually; in addition, offerings of veterinary drugs at concentrations consistent with formulated feeds as testing items were increased in frequency and variety from approximately 1 or 2 drugs in 2 or 3 testing rounds annually to approximately 3 to 5 drugs in a minimum of 6 rounds annually.

As of February 2018, all test offerings were fully ISO/IEC 17043 accredited. This program is financially self-sustaining, owing to the high number of participating laboratories.

Publications Supporting Data Defensibility

In October 2016, APHL’s Data Acceptance Work Group, which consists of federal, state, and association food testing partners, published a white paper (referred to as the Data Acceptance White Paper) on best practices for submitting food and feed testing data from state and local laboratories to regulatory agencies.¹³ Government human and animal food testing laboratories often face unique situations that result in producing data outside the scope of their accreditation, such as testing performed during foodborne outbreak investigations, emergency responses, or in rare circumstances (eg, foreign objects in manufactured foods). The Data Acceptance White Paper provides guidance for building a robust quality management system to meet the needs of federal and state regulatory programs and to support acceptance of laboratory data regardless of whether the testing was performed under the laboratory’s scope of accreditation. An AAFCO-led working group of state, federal, and industry partners produced the 2014 AAFCO Quality Assurance/Quality Control Guidelines for Feed Laboratories,¹⁴ which is the official guidance document for animal food testing laboratories and a supplement for implementing ISO/IEC 17025 accreditation.

The quality of laboratory data is only as good as the primary sample collected and the test portion (the portion of the sample selected for testing). Data from various laboratories or organizations can never be equivalent if sampling practices are not equivalent. To help ensure equivalent sampling practices, the FDA funded a working group to develop sampling guidelines, which led to the publication of 13 peer-reviewed articles in a special section on sampling food and feed materials in the Journal of AOAC International.^15-27 After additional study and effort, the AAFCO/AFDO/APHL Sampling Working Group released a national guidance on sampling, GOODSamples: Guidance on Obtaining Defensible Samples,²⁸ in October 2015. This working group consisted of representatives from the FDA, state human and animal food testing programs, state laboratories, and industry; inspection staff members; and subject-matter experts. GOODSamples outlines a systematic approach to sampling based on the theory of sampling²⁹ to ensure that analytic data are representative and defensible (Box 2). GOODSamples principles and concepts were further elaborated for laboratory sampling processes by a subsequent AAFCO/AFDO/APHL working group that released a companion guidance on test portions, GOOD Test Portions: Guidance on Obtaining Defensible Test Portions,³⁰ in 2018.

Box 2.

Characteristics of a good food testing laboratory sample, from the theory of sampling²⁹

A sample is representative if it (1) is correct (ie, bias is deemed to be negligible) and (2) has a sufficiently small random error (controlled by appropriate mass and sufficient number of increments) to meet needed confidence to meet data objectives or sample quality criteria.

A sample is defensible if the scientific quality and evidentiary integrity is sufficient to justify the action or outcome (eg, data, decision).

Laboratory testing performed by a competent workforce with standardized training supports data defensibility. In collaboration with the International Food Protection Training Institute, AFDO’s Governmental Human and Animal Food Laboratory Curriculum Framework Workgroup, which consists of subject-matter experts from federal, state, and local laboratories, along with the Canadian Food Inspection Agency and APHL/AFDO/AAFCO staff members, drafted a curriculum framework and standardized definitions³¹ to guide the development of competency statements for laboratory staff members at various professional levels. The framework is intended to standardize training for laboratory staff members, allow laboratories to identify gaps in professional development, and help laboratories create personalized learning paths for employees; it is analogous to a similar framework developed for regulatory personnel, the National Curriculum Standard.³² Although adhering to these guidance documents is not required for ISO/IEC 17025 accreditation, the documents’ recommendations help laboratories strengthen their quality management systems and develop confidence in their ability to create accurate and defensible data through correct sampling procedures and technical competence.

Meetings and Training Events

In-person meetings and training opportunities are highly effective for communicating lessons learned and best practices. Attendees can discuss common issues and brainstorm solutions. Since 2015, the annual Governmental Food and Feed Laboratory Accreditation Meeting³³ has provided a forum for human and animal food testing laboratorians to learn best practices and connect with colleagues about current and emerging food safety issues. Held in conjunction with the Manufactured Food Regulatory Program Alliance meeting, this meeting helps laboratorians network with their state regulators to discuss current issues and collaborate on possible solutions.

The GOODSamples concepts and content were disseminated through many presentations and 1-day workshops held between 2015 and 2018. In addition, AAFCO delivered 6 focused, in-depth, 4-day training events and 6 state sampling pilots based on GOODSamples concepts.^34-37 These training events and pilots brought together various food safety professionals (ie, inspectors, laboratorians, quality managers, management) who learned best practices for sampling to improve accuracy in detecting contaminants and support defensible regulatory decisions for any sampling objective.

These meetings and training events provide valuable opportunities for participants to connect with colleagues and learn from others who are further along in the accreditation process. Attendees can also interact with federal partners and state program staff members, which provides opportunities to collaborate and solve common food safety–related issues.

Lessons Learned and Implications for Public Health Practice

Since the inception of the cooperative agreements, 37 laboratories have achieved or expanded the scope of their ISO/IEC 17025 accreditation using FDA funds and association resources, training, and support. In January 2017, APHL provided details to the FDA on the effect of the ISO cooperative agreement programs and requested $4.5 million in continued funding for the programs.³⁸ Recognizing the value of its initial investment and the ongoing need to support laboratory accreditation as a component of an Integrated Food Safety System, the FDA awarded additional 2-year funding³⁹ in September 2017 to the initial cohort of funded laboratories and to APHL to continue supporting accreditation efforts.

Accreditation brings immediate and long-term benefits to a laboratory and its customers by requiring the development and implementation of a robust quality management system whose technical requirements include competency of laboratory personnel, validated laboratory procedures, and quality control measures. The cooperative agreement programs have fostered partnerships and created networking opportunities among state laboratories and between these laboratories and regulatory agencies. For example, in October 2014, state officials in Massachusetts investigated an outbreak of Campylobacter associated with raw milk from a local farm.⁴⁰ At that time, the Massachusetts State Public Health Laboratory did not have the capability to test for Campylobacter in food or milk. Through the cooperative agreement, the Massachusetts laboratory knew that the New York Department of Agriculture and Markets Food Laboratory could test for Campylobacter in raw milk. Because the New York laboratory was ISO/IEC 17025 accredited, Massachusetts state officials accepted its results and took action to remove the raw milk from commerce.

The Florida Department of Agriculture and Consumer Services, Bureau of Food Laboratories collaborated with state regulators to devise a joint sampling plan that targeted high-risk manufactured food items linked to recent recalls.⁴¹ In January 2017, just 1 month after being added to the sampling plan, the Florida laboratory, using an ISO/IEC 17025-accredited method, discovered Listeria monocytogenes in a frozen vegetable mix product. The FDA accepted the laboratory’s test data and worked with the manufacturer to issue a voluntary product recall.

The accreditation process can be met with doubt and hesitation from employees who are resistant to change. However, accreditation is becoming the standard of practice in laboratories across the nation and around the world. For state human and animal food testing laboratories, gaps in capacities remain, including the need to institutionalize the concepts of GOODSamples and GOOD Test Portions, improve laboratory handling and preparation procedures, improve the processes for reviewing and uploading data to the FDA, and develop a standard national laboratory curriculum to promote a consistently competent workforce.

In 2017, APHL surveyed 49 laboratories involved in the cooperative agreements and laboratories receiving assistance from APHL’s accreditation consultant to determine the effect of the cooperative agreements.⁴² Of 36 laboratories that responded, 28 used APHL’s training resources, and 8 worked with a regulatory program that adopted sampling practices based on the GOODSamples theory. In addition, 32 laboratories reported subscribing to AAFCO’s Proficiency Testing Program, and representatives from all 32 laboratories who attended association-held meetings found them to be helpful for training and networking purposes.

The deliverables implemented under the cooperative agreements have likely helped improve data defensibility for governmental food testing and expanded the scope of accredited food testing by government laboratories in the United States. Although the scope of this expansion may decline with decreased funding, the lessons learned from these cooperative agreements about the importance of quality assurance will remain relevant. The investment in laboratory accreditation continues to provide benefits by protecting public health and allowing both state and federal regulatory agencies to act quickly on data provided by accredited laboratories when the data suggest a problem in the food supply. Government laboratory accreditation strengthens the food safety system and expands the nation’s capacity to perform regulatory food testing. Working together, laboratories and their partners can achieve a level of quality and efficiency that supports the protection of human and animal food products through data and information sharing.