Progress in the Accreditation of Anthropology Laboratories

Abstract

While crime laboratories are commonly accredited under programs utilizing international standards, options for forensic anthropologists to do the same were limited, until recently. The American National Standards Institute-American Society for Quality (ANSI-ASQ) National Accreditation Board (ANAB) and the American Association for Laboratory Accreditation (A2LA) both offer accreditation programs for forensic anthropology services using either the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 17025 or ISO/IEC 17020 standard. The significant efforts made to specifically include forensic anthropology, and also forensic pathology, in these national programs demonstrate the importance for all practitioners in the field of forensic science to develop and maintain quality assurance programs consistent with international standards. Among the requirements for quality assurance is validation of methods, a practice that was previously identified as needing improvement within the forensic anthropology community.

Introduction

Following the publication of the 2009 National Academy of Sciences (NAS) Report: Strengthening Forensic Science in the United States: A Path Forward (1), establishing quality assurance programs and achieving accreditation became a priority for forensic anthropology practitioners. In light of the focused response within the field, Syracuse University hosted a conference in 2011 entitled “Beyond the NAS Report,” during which the current and future impacts of the report were examined. One aspect discussed was the need for accreditation of anthropology laboratories. While it has been several years since the NAS recommended mandatory accreditation for laboratories performing forensic analyses, and decades since many crime laboratories had already achieved accreditation, very few forensic anthropology laboratories have accomplished this objective. The topic, however, remains relevant today for multiple reasons: 1) good scientific practice requires accuracy and repeatability, neither of which is possible without transparency, and accreditation invites the kind of transparency that facilitates accuracy and repeatability via detailed communication of procedures, error rates, etc.; 2) the public expects unimpeachable service from forensic science laboratories and practitioners; and 3) delivery of a deficient service or lack of transparency can result in poor public perception, fair or not. The latter occurrence will further result in loss of either customer base (for privately-owned agencies) or funding (for government-owned agencies). Therefore, it is imperative that agencies allocate resources to ensure solid scientific practice to protect their public image and maintain the public's trust. These resources should ultimately support a quality management system. By adhering to established standards for quality service, a forensic anthropology laboratory can demonstrate its credibility to the community it serves. This process will become more urgent as the federal government, and all of the public and private practitioners it has engaged, gains momentum in its efforts to advance the forensic sciences through entities like the Organization of Scientific Area Committees (OSAC) and the National Commission on Forensic Sciences (NCFS).

Discussion

Forensic anthropology laboratories face several obstacles that complicate the pursuit of accreditation. Lack of meaningful precedents and guidance, insufficient human resources, tight funding, unsupportive management, and feelings of disconnect from the rest of the forensic domain are among the reasons anthropologists have cited for not pursuing accreditation (2). One of the main reasons cited is that existing accreditation programs are unsuitable or an improper fit for forensic anthropology. Until recently, no program named anthropology under the disciplines it accredited. Whereas the National Association of Medical Examiners (NAME) accredits forensic pathology offices, none of the professional anthropology organizations accredit forensic anthropology laboratories.

There has been considerable progress on at least some of these fronts. The Organization of Scientific Area Committees was developed in 2014, and is tasked with the development and maintenance of standards and best practice guidelines for many of the forensic disciplines, including anthropology. The Anthropology Subcommittee of the OSAC has leveraged the work completed by the preexisting Scientific Working Group for Forensic Anthropology (SWGANTH) to develop best practice recommendations for forensic anthropology regarding practitioner certification, methodology, and reporting of analytical results. As far as precedent, until recently the Defense POW/MIA Accounting Agency (DPAA), formally known as the Joint POW/MIA Accounting Command Central Identification Laboratory (JPAC-CIL), was the only forensic anthropology laboratory with any type of accreditation. The DPAA developed a quality assurance program that facilitated the laboratory's accreditation by the American Society of Crime Laboratory Directors/Laboratory Accreditation Board (ASCLD/LAB) in 2003. The laboratory was re-accredited under the ASCLD/LAB-International Program in Crime Scene and Trace Evidence in 2008. At the time, accreditation of anthropology laboratories was only available under a trace laboratory rubric.

Standards and Accreditation Bodies

The International Organization for Standardization (ISO) is an independent organization headquartered in Geneva, Switzerland that develops and publishes international standards. These international standards are used to ensure that products and services being sold are safe, efficient, and of high quality. From agriculture to engineering, from automotive to healthcare, standards exist for every type of industry. These highly regarded standards translate across continents because experts from around the world develop them. In addition to subject matter experts, opinions from consumer associations, academics, and government officials are taken into consideration when creating these standards.

The original purpose for establishing industry standards was to facilitate international trade. It brought opportunities for developing countries to conduct business in markets previously unattainable to them. Not only have ISO standards promoted fair trade, but they have also led to great economic expansion during the past 70 years. Currently, there are over 21 000 international standards being utilized across 162 countries (3). While some ISO standards have been incorporated into legislation to ensure businesses are mindful of public health and safety, other ISO standards have been added to voluntary accreditation programs for testing, inspection, and certification bodies.

In 2013, Forensic Quality Services (FQS), now known as the American National Standards Institute-American Society for Quality (ANSI-ASQ) National Accreditation Board (ANAB), added anthropology as a recognized field in its accreditation program for forensic agencies. The ANAB is a U.S. entity that provides accreditation to certification bodies and inspection bodies. The ANAB accreditation programs geared towards forensic agencies incorporate ISO standards, along with field-specific supplemental criteria. The American Association for Laboratory Accreditation (A2LA) is another accreditation body in the United States that offers accreditation for laboratory-related services, also using ISO standards. A2LA's Forensic Identification Accreditation Program is now open to disciplines involved in recovering and identifying human remains, including forensic anthropology, forensic biology, forensic odontology, and fingerprint collection and analysis.

There are currently two ISO standards being used by accreditation bodies to accredit forensic science agencies: ISO/IEC 17025 (general requirements for the competence of testing and calibration laboratories), and ISO/IEC 17020 (conformity assessment - requirements for the operation of various types of bodies performing inspection). ISO/IEC 17025 has a long history of broad utilization in the accreditation of crime laboratories and is more appropriate for laboratories whose experts report results based on data generated by analytical equipment. ISO/IEC 17020 has more recently gained popularity with crime scene investigation units and is more appropriate for anthropology laboratory accreditation because it is focused instead on analyses that require an expert's professional judgment.

ISO/IEC 17025 contains requirements that testing and calibration laboratories must meet in order to demonstrate that they operate an organized management system, are technically competent, and are able to generate technically valid results. ISO/IEC 17020 is a set of standards with many of the same quality management principles but focus technically on “inspection activities,” which is another term for conducting an investigation or examination. This can be applied to a crime scene unit, a latent print unit, a digital media laboratory, and a forensic anthropology laboratory. The main difference between ISO 17025 and ISO 17020 is that the former applies more to analytical testing, while the latter pertains to functional examination.

In August 2015, the Harris County Institute of Forensic Sciences (HCIFS) Forensic Anthropology Division became the first forensic anthropology laboratory to be accredited under ANAB's ISO/IEC 17020 inspection body program. The accreditation required collaboration by management, anthropology staff, and quality assurance staff. Among the areas assessed under this program are an agency's personnel, facilities, administration, examination methods, procedures, records, and management system. Therefore, agencies accredited under this program have objective proof of a quality operation, as they must continuously meet or exceed the international standards for quality examinations. As of yet, no other anthropology laboratory is accredited through this program; however, the program is still new to the anthropology community and preparation for accreditation can take several years.

Validation of Methods

Accreditation requires compliance with several hundred criteria, the most important of which are the use of validated methods and written procedures for performing said validations. This is an area that is particularly difficult for seasoned practitioners to embrace because, more than likely, they already use methods that have been widely accepted within the community. Once qualified to work independently, anthropologists usually adopt the same methods and equipment used by their mentors and/or absorb methods from the peer-reviewed literature. Thus, analysts often believe the validation is already completed or find it not applicable if a method has been well-examined in the literature. It is only through understanding of the quality assurance process that one realizes adopting and implementing a popular method is not enough.

Compliance with method validation requirements can be subdivided into three components: validation, verification, and performance monitoring. Although these terms may be used interchangeably, or with different meanings in other disciplines, they serve distinct purposes for assessing the validity of a method used by a forensic examiner. Validation, occasionally referred to as a developmental or primary validation, provides objective evidence to confirm that requirements for the method's specific intended use or analytical application have been fulfilled (4). By conducting and analyzing data from a set of studies, the efficacy and reliability of a technique or procedure can be established. Validation differs from verification in that verification provides objective evidence to confirm that specified requirements set forth by the primary validation have been fulfilled (5). Also known as an internal validation or secondary validation, a laboratory may conduct a verification to demonstrate it is capable of reliably and precisely performing a method for its intended use as seen in the literature or at another laboratory.

This is not to be confused with performance monitoring, which is a quality check of a procedure or instrument to ensure it performs as expected. Quality control samples or reference standards are often used to check the performance of an existing procedure or piece of equipment that underwent validation and verification.

In order to meet the accreditation requirements encompassing validation, practitioners should decide which methods would be subject to each of the above-mentioned measures. A method previously validated by an outside party does not necessarily need to be revalidated by the lab seeking accreditation. As long as the procedure for the method validation can be located, the laboratory is only required to verify the analytical method can be used correctly within its own setting. The procedure and equipment used for that method must then be subject to intermittent performance checks for quality assurance purposes. Performance checks ensure that a random change affecting the method does not go unnoticed.

As previously mentioned, anthropology practitioners are likely already employing validated methods; however, in-house verification is warranted for guaranteeing the appropriateness of their use for fulfilling the service needs of the laboratory. Not only will this practice contribute to the reliability of examination results, but it will also bring forensic laboratories one step closer to complying with accreditation standards.

A newly developed method, on the other hand, must be validated in-house if it has not been validated elsewhere. Documented validation studies should include, where applicable, precision and accuracy, sensitivity and stability, reproducibility, and population data. The validation must produce data that demonstrates the method is capable of successfully performing at the level of its intended use and to identify its limitations under normal operating conditions (6). Previously validated methods must also undergo a new validation when changes are made to the procedure or its application. Although not as extensive as the developmental validation, the new validation must be sufficient to demonstrate that the method continues to produce reliable results despite the modification. Procedural modifications requiring additional validation are typically changes made to either improve the method or extend its use beyond that for which it was originally validated. Examples include changes in materials or equipment used or the introduction of an automated step.

Software programs are also subject to validation, verification, and performance checks if casework depends on their data output. The need for in-house programs and templates to be properly tested before use is unquestionable, considering the specialized software used for statistical calculations and new efforts to render documentation of laboratory functions paperless. Therefore, known sets of data should be generated to adequately test these software programs and the procedures utilized by staff that runs them. Analytical programs that involve a significant user interface, allow manipulation of data, and make decisions determining the direction of the analysis require more complex validation procedures. Forensic laboratories that fail to validate or verify their software programs, automated report templates, and macroinstructions risk introducing errors into their analyses. An accredited laboratory should have a formal procedure for validating and implementing programming changes. The strategy used by the HCIFS to meet this requirement was presented at the National Institute of Science and Technology (NIST) International Symposium on Forensic Science Error Management in July 2015 (7). Given the rate of technological advances, programming validation will continue to be an important component of laboratory quality assurance.

Conclusion

The examination and reporting process of forensic anthropologists are more similar in nature to that of forensic pathologists than crime laboratory analysts. They rely less on data from automatic instrumentation and more on findings and interpretations from direct observation. Due to the often subjective and experienced-based nature of analyses in forensic pathology and anthropology, it is easy to see how practitioners may not feel the necessity to adopt all quality assurance standards recommended for a laboratory. However, quality systems are adaptable and should be developed to address the specific needs of each discipline. Method validation, for example, is an area that warrants discipline-specific instruction. The forensic anthropology community will benefit from publication of the detailed guidance being generated by the Anthropology Subcommittee of the OSAC.

In May of 2016, ANAB announced its coalition with the National Association of Medical Examiners, effectively adding the ISO 17020 standards to the NAME accreditation program. It was determined that the ANAB ISO 17020 program applies to forensic pathologists just as it applies to forensic anthropologists; both types of experts use their training and experience to examine remains and formulate conclusions based on their observations. While these changes to the NAME accreditation program may seem intimidating, the intention is to advance, along with other forensic disciplines, towards uniformity and higher quality. All forensic practitioners have the responsibility to ensure their opinions and testimonies are based on sound science, ethical practices, and transparent operations. The ISO standards have always fostered these objectives, and now avenues exist for forensic practitioners in the medicolegal setting to become accredited to those standards.