The invisible crisis facing forensic providers in the United States

Abstract

There is a dissonance between the public perception of forensic practice in the United States and reality. Portrayed as infallible and universally available, the truth is that the awkward evolution of forensic practice has created fissures in its foundations, affecting both the quality and quantity of these critical services. These fissures have only widened over time, creating problems that require creative solutions.

Within the United States, every level of government provides forensic services, and there is no overarching authority over forensic providers. The recent reevaluation of the Chevron deference doctrine by the Supreme Court ensures that this is unlikely to change in the foreseeable future. Under Chevron deference, courts would defer to regulatory agencies interpretations when laws were ambiguous. This new ruling transfers that responsibility back to the courts, making the creation of a new regulatory agency over forensic providers even less likely. Thus, any meaningful change to forensic practice must come from within the profession. As unlikely as this may seem, it has occurred before.

This perspective allows us to see the hidden challenges forensic providers face and propose potential solutions that practitioners can implement to increase the quality of their work and the resilience of the system they operate within. By evaluating how forensic practice has implemented change in the past, this paper proposes solutions that have worked before and expands on others from different industries with similar challenges.

1. Background

In 1972, during the Plenary Session of the 25th annual meeting of the American Academy of Forensic Sciences (AAFS), Duayne Dillon called the FBI laboratory an obstacle to the broader spread of criminalistics in the United States [1]. Even though Briggs White, the Director of the FBI laboratory, was in the audience, Dillon's statement wasn't meant to be antagonistic. Dillon was taking advantage of a unique opportunity. Longtime FBI Director J. Edgar Hoover had just died. Hoover was not known for sharing FBI resources during his 37 years at the helm of the agency, and Dillon was highlighting how the FBI could become a resource for local crime laboratories instead of an impediment to them.

Dillon's comment had the intended effect. Soon after the AAFS meeting, the FBI opened training opportunities for forensic providers outside the FBI and met with 30 laboratory directors across the U.S. This meeting catalyzed forensic laboratory directors to form an organization to discuss challenges and share solutions. The American Society of Crime Laboratory Directors (ASCLD) was officially established in the fall of the next year [2].

As forensic professionals started to discuss quality, the Law Enforcement Assistance Administration (LEAA) began exploring ways to assess it. The LEAA's first attempt to develop a proficiency testing program turned into a wake-up call for the entire profession. Their 1978 report exposed glaring deficiencies in the nation's forensic laboratories with varied levels of proficiency requiring remediation through immediate, comprehensive retraining [3].

Since there is no overarching authority over forensic providers in the United States, the nascent organization of crime laboratory directors recognized that the issues highlighted in the LEAA report had to be addressed and created a committee on laboratory evaluation and standards. Their goal was to develop minimum standards for forensic laboratories and address the concerns of the 1978 LEAA report. The National Association of Medical Examiners (NAME) had developed an accreditation program for Medical Examiner's Offices in 1974 [4], but there was no similar program specifically for forensic laboratories.

After three years, the committee split off from ASCLD forming its own organization to accredit forensic laboratories to the standards they had developed. They called the new organization the American Society of Crime Laboratory Directors-Laboratory Accreditation Board, or ASCLD/LAB [5].

While ASCLD/LAB operated separately from ASCLD, there was an overlap between the two organizations. Any changes to the accreditation standards needed to be approved by the ASCLD/LAB delegate assembly, composed of ASCLD members. The ASCLD/LAB board, also composed of elected members of ASCLD, made all final accreditation decisions.

The Illinois State Police Laboratory was the first forensic laboratory accredited by ASCLD/LAB in 1982 [6]. Overall, the forensic community was slow to adopt accreditation. Most forensic providers in the United States operate under parent agencies with different prime mandates (i.e…, law enforcement agencies or health departments), and it was difficult for forensic providers to make the case to agency decision makers to adopt more rigorous standards when there was no external requirement to do so.

This dramatically changed after a law was enacted that required forensic laboratories to be accredited to access the national DNA databank [7]. The creation of the national DNA databank also served as the initial motivation for New York to adopt an oversight body over its forensic laboratories with corresponding accreditation requirements. Since then, a number of jurisdictions have adopted a form of oversight body or commission with some level of accreditation requirement [8].

14 years after ASCLD/LAB began accrediting forensic laboratories, the American Board of Forensic Toxicology (ABFT) began offering an accreditation program specifically for forensic toxicology laboratories and accredited their first laboratory in 1997 [9]. These first forensic accreditation programs had the advantage of being heavily connected to the disciplines they assessed, and in ABFT's case, highly specific to their discipline. However, this connection also raised questions about objectivity. These legacy programs had another significant disadvantage: They were not recognized beyond their organizations.

Up until this point, there were no internationally recognized standards for forensic practice. The International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) developed a guide (ISO/IEC Guide 25), but guides are not as advanced as standards. In 1998, the ISO/IEC released the first international standard that could be applied to some areas of forensic practice. ISO/IEC 17020, titled “conformity assessment - requirements for the operation of various types of bodies performing inspection” seemed well-suited for providers that focused on the recognition, documentation, and collection of evidence such as crime scene processing. The following year, ISO/IEC released ISO/IEC 17025, titled “general requirements for the competence of testing and calibration laboratories” which was a more appropriate international standard for forensic providers that analyze physical evidence.

For a short time, the National Forensic Science Technology Center (NFSTC) developed an international accreditation program, and, in 2001, the Georgia Bureau of Investigations became the first forensic laboratory in the United States to be accredited to the ISO/IEC 17025 standard [10]. In 2004, ASCLD/LAB offered accreditation under two different program tracks, a traditional legacy program and an international program. By 2014, ASCLD/LAB was only offering the international program [11].

It's difficult for a private organization to maintain an international accreditation program specific to an industry that is relatively small compared to other fields of endeavor with more cohesive regulatory and professional standards. As a result, the move to international accreditation programs inadvertently fostered the migration of forensic accreditation programs away from forensic practitioners toward organizations offering accreditation services to multiple industries.

2. The problems

2.1. Understanding accreditation

When a forensic provider achieves initial accreditation, press releases often describe them as “among the best” [12] or at “the pinnacle” [13] of their profession. However, accreditation only means that the provider has the most basic components of a quality system in place.

Accreditation is a critical first step toward achieving quality and reliability, but it's only the first step. It's akin to registering for a race rather than finishing it in first place. This misperception may be a natural one. Since the industry lacks a universal requirement to become accredited, providers who invest the extra time, money, and effort to undergo the process are moving well beyond many of their peers.

2.2. The Tradeoff associated with moving to international accreditation vendors

There were both advantages and disadvantages to forensic practice moving away from legacy accreditation programs. The international program introduced critical concepts such as root cause analysis, corrective action, and measurement uncertainty that were not part of the legacy accreditation programs. The professional distance from the fields these programs assessed helped to minimize concerns about conflicts of interest, but there were also negative aspects to this increased distance.

Under the ASCLD/LAB program, the standards were developed and maintained by ASCLD members, and accreditation decisions were made by a board elected among the ASCLD membership. These connections were lost, and as a result, many of the standards that industry professionals felt were important were downgraded to conform to international standards. Requirements for administrative review were removed, and those for technical review were diluted. The qualifications for an analyst, Quality Assurance Manager, and Director were all more robust under the legacy programs, and the requirement for providers to have a Technical Leader for each category of testing was removed during the transformation to the International Program.

2.3. The consolidation of accreditation providers

There was also consolidation within the private accreditation vendor market. Like ASCLD/LAB, NFSTC started as an offshoot of ASCLD [14]. NSFTC soon began to also offer an international accreditation program through Forensic Quality Services (FQS) which was a division within NFSTC. FQS was then split into a subsidiary to provide accreditation services [15]. In 2011, the ANSI-ASQ National Accreditation Board acquired FQS [16]. In 2016, ANAB merged forensic operations with ASCLD/LAB [17]. That same year, ANAB formed a “strategic alliance” with NAME and ABFT to administer their respective accreditation programs [18,19]. Finally, in 2018, ANSI-ASQ acquired full interest in ANAB, creating the ANSI National Accreditation Board (ANAB) [20].

While the landscape of international accreditation vendors has changed over time, the most current snapshot is through the International Laboratory Accreditation Cooperation (ILAC). The ILAC is a global organization for accrediting bodies. International accreditation vendors are required as signatories to the ILAC's Mutual Recognition Agreement to provide either ISO/IEC 17020 or 17025 accreditation services within the United States.

The ILAC's website has a searchable index [21]. Using that searchable index at the time of this review and narrowing the “economy” to the United States, seven accreditation vendors were identified. AIHA Laboratory Accreditation Programs, LLC (AIHA LAP, LLC), the American Association for Laboratory Accreditation (A2LA), the ANSI National Accreditation Board (ANAB), International Accreditation Services, Inc. (IAS), NAC National Accreditation Center LLC (NAC), National Voluntary Laboratory Accreditation Program (NVLAP), and Perry Johnson Laboratory Accreditation, Inc. (PJLA). Four include scopes covering ISO/IEC 17020, six include scopes covering ISO/IEC 17025 for calibration, and all seven include scopes covering ISO/IEC 17025 for testing.

The ILAC MRA search results also contain links to the agencies accredited by that vendor. Tabulating the data from each of the seven vendors provides a more realistic picture of how many vendors are accrediting forensic laboratories in the United States and how distributed the workload is over the seven vendors.

Of the seven different accreditation vendors evaluated at the time of this review, only four were actively accrediting either government or private forensic providers. 90 % of the accredited forensic providers were government, and the work was not evenly distributed among vendors. Two of the seven vendors (A2LA and ANAB) are accrediting over 99 % of the forensic providers in the United States, and one vendor (ANAB) was accrediting 596 of the 633 listed.

2.4. Regulating government operations

Lacking any uniform oversight over forensic providers, accreditation has become the default regulatory authority over the forensic industry. While there are some jurisdictions that also “accredit” some of their forensic providers, these accreditations rely heavily, if not exclusively, on the accreditations provided by these private vendors. This creates a less-than-optimal situation where private vendors have become the de facto regulators of high-consequence government operations. This situation is further complicated by the consolidation of the private vendor market. Currently, one private company (ANAB), is essentially the default regulator over forensic practice in the United States.

Unlike federal, state, and municipal governments, private companies are not obligated to provide a service or any justification for changing their business model. Should the single private company that accredits over 99 % of the forensic providers in the United States decide to cease their forensic operations, the consequences for the entire industry could be dire and there would be no recourse.

Illustrating this point, the internet retail giant Amazon closed seven warehouses in the province of Quebec, Canada this year shortly after local workers voted to unionize one of the warehouses [22]. Amazon has denied that the closures are related to the recent unionization [23], but there is no way of knowing what prompted the move since private companies are not subject to the transparency of governmental operations. What is not in dispute is that a private company made a unilateral decision resulting in seven warehouse closures, prompting the loss of thousands of jobs and the termination of service to an entire region of the country. A similar action by the leading forensic accreditation provider could have devastating consequences.

2.5. Current accreditation mandates

The first real accreditation mandate came indirectly through the passage of the DNA Identification Act. To participate in the National DNA Databank, DNA laboratories had to be accredited [7]. While the accreditation mandate did not apply to other types of forensic testing, the ASCLD/LAB accreditation program which was essentially the only program at the time, required all eligible sections of a laboratory to undergo the accreditation process.

The ability to use DNA as an investigative tool acted as a significant incentive for the agencies that host forensic services and allowed them to overcome any reluctance they might have had about undergoing the accreditation process. As a result, the quality of all the services they provided increased.

Since then, new international accreditation standard vendors have taken a different approach. It's not uncommon in other industries to only accredit a single process. Accreditation vendors often allow applicants to decide which processes benefit from accreditation. New ISO/IEC 17025 accreditation vendors adopted this approach, allowing forensic laboratories to only accredit their DNA sections so they could gain access to the National DNA databank.

In industry, this approach works. The quality of critical items is primarily driven by market forces. If a manufacturer produces subpar critical products, customers won't buy them, and the manufacturer will either increase the quality of the product to gain the market share back or stop manufacturing the product as demand declines.

These considerations do not apply to forensic practice. Since forensic services are primarily run by governmental agencies, they are insulated from market forces that help regulate critical items in industry. Instead, jurisdiction drives who does the work, often irrespective of the quality of the work performed. This process can also be complicated when agency decision makers are not subject matter experts. In these situations, cost and effort often outweigh the potential of increased quality. This creates a dangerous loophole.

The failure of the Detroit Police Crime Laboratory highlighted the risk of this approach. The agency's decision makers elected to hire an accreditation provider that allowed them to select which sections of their laboratory would undergo accreditation. They opted to only accredit their Biology section, the minimum requirement to access the national DNA database. In 2008, problems with the laboratory's unaccredited firearms unit prompted a broader audit and the closure of the entire laboratory. At the news conference announcing the lab's closure, the Wayne County Prosecutor described problems in the laboratory as “… so severe as to demonstrate a systemic problem in all disciplines … [24]”

Several jurisdictions have accreditation mandates, but there is no standard definition across these different jurisdictions specifying to whom the mandate applies. Even when there is clarity within a given jurisdiction on accreditation applicability, there is no guarantee the mandates will be uniformly enforced.

As an example, in 1994, New York was one of the first states to implement such a mandate. The enabling legislation also created a Commission on Forensic Science (CFS) that was to be chaired by the head of the Division of Criminal Justice Services, whose agency would also provide the staff support to the CFS [25] and manage the state's accreditation program. Despite DCJS's pivotal role in enforcing the state's accreditation mandate, the agency also operated an unaccredited latent print comparison laboratory for over two decades. DCJS's Latent Print Comparison Laboratory was first accredited in 2016 [26], 22 years after the accreditation requirement was introduced.

Digital evidence laboratories are also required to be accredited under New York's mandate. Currently three [[27], [28], [29]] of the 22 [30] laboratories that are accredited by New York State list Digital as part of their scopes. However, there are significantly more forensic digital laboratories in the state that meet the definition as outlined in N.Y. EXC. LAW § 995 that remain unaccredited, many operated by the state. One [31] is in the state crime laboratory building, merely a few feet away from the accredited Crime Laboratory [32]. It's noteworthy that an executive from that agency has been a member of the CFS since its inception, offering evidence that this decision was not an oversight.

In 2015, the National Commission on Forensic Science (NCFS) approved a recommendation for Universal Accreditation of forensic service providers [33]. Later that same year, the Attorney General issued a memorandum instructing all federal departments that provide forensic science services (except digital evidence) to become accredited by December 2020 [34]. It also directed all Department Attorneys to use “… whenever practicable, accredited forensic testing entities.” It is unclear what, if any, progress has been made with either of these directives. At the very least, the “whenever practicable” language creates a substantial loophole.

2.6. Digital evidence accreditation

In forensic science, examination of an individual's digital footprint offers valuable insight into where the individual has been, what they have done, with whom they have communicated, and by extrapolation of their search data, what they might have been thinking. Some aspects of digital evidence analysis are used in nearly every major investigation, often more frequently than fingerprint or DNA evidence. Yet comparatively few digital evidence providers have undergone accreditation, and some mandates may even specifically exclude them [34].

Many lawmakers are reluctant to impose any barriers on the collection or analysis of digital evidence for fear of hampering the effectiveness of law enforcement, but this concern carries its own risk. Without standard operating procedures, trained examiners, validated techniques, and quality oversight, the reliability of this critical evidence may be in question.

Illustrating this point, during one high-profile murder trial, the state presented digital evidence that a computer visited a website for “chloroform” 84 times, a fact the defense claimed was used by the prosecution to try and establish premeditation. It turns out that the computer had visited “Myspace” 84 times [35]. A significant discrepancy that highlights that no forensic service can be deemed reliable without quality oversight in place.

2.7. Forensic science service providers

Currently, there is no record of all the different public agencies that provide forensic services in the United States. The most recent data from the Bureau of Justice Statistics [36] describes “… 326 publicly funded forensic laboratories and multilab systems …”, but this value is just a fraction of agencies that document, collect, and analyze evidence from the start of the investigation. Illustrating this point, one publication estimates that the number of agencies performing forensic services in New York is likely ten times the number of that state's accredited laboratories [37].

Some of the most recent efforts to improve forensic practice have acknowledged that science doesn't start at the laboratory door, and all providers should be accredited [33,38]. Years after those recommendations, unaccredited forensic providers are still being found to have no standard operating procedures or even basic quality systems in place [39].

2.8. Voluntary compliance

Forensic practice is the only high-consequence industry that relies primarily on voluntary compliance to ensure quality. Lacking any uniform mandates to become accredited or adopt newly developed and approved standards [40], it is up to the heads of agencies that provide forensic services to decide if they will comply or not.

2.9. Resistance to change

In 2006, the National Academy of Sciences (NAS) was awarded over 1.4 million dollars in response to a National Institute of Justice grant seeking to maximize the capabilities of the forensic science community [41]. Three years later, they released a comprehensive 328-page report with specific recommendations to improve forensic practice in the United States [42]. Nearly two decades later, none of the NAS recommendations have been fully implemented. Subsequent national efforts have been met with a similar level of success.

2.10. Institutional support

It is widely known that Dr. Edmond Locard started the first Police Crime Laboratory in Lyon in 1910. What is less well known is how poorly the venture was supported. Dr. Locard was provided with only an office in the attic of the courthouse, equipped with an old Bunsen burner and two straw chairs [43]. Despite his massive contributions to the local Criminal Justice system and the field of forensic science overall, he never received an annual budget to run the laboratory, a salary for his work, or any type of pension after he retired [44].

It's been more than a century since Dr. Locard demonstrated the transformational power of scientific evidence to aid in both the investigative and adjudicative processes, yet funding remains anemic.

3. Creative solutions

To develop creative solutions to address the unique challenges facing forensic practice in the United States, it is necessary to invert the current paradigms that are causing the problems, changing current weaknesses to future strengths. While these challenges are unique to forensic practice, potential solutions are available through other industries that have already demonstrated their effectiveness.

For example, the federal government sets the emission standards for vehicles manufactured or sold in the United States. Unsatisfied with those standards, the State of California has set more rigorous emission standards. Since the vehicle market in California is so lucrative and it would be too difficult to exclude this market or to manufacture vehicles for two different emission standards, vehicles are being manufactured to higher emission standards than those set by the federal government.

It's possible to utilize a similar model to address several of the problems described in this perspective. Recently, the National Association of Forensic Science Boards (NAFSB) formed. A non-profit organized out of North Carolina, according to the organization's website, the NAFSB seeks to “disseminate and exchange best practices, research, expertise, data and lessons learned among state-level forensic science boards and commissions governing the forensic science community [45].”

Currently, 21 states and the District of Columbia have some form of oversight body or commission in place with varying levels of authority [8]. At the time this review, state and local government forensic providers from these jurisdictions make up over 44 percent of the forensic providers accredited by ANAB in the United States. While this is not the entire accreditation market, it is enough to leverage change. The most recent data from the California New Car Dealers Association [46] shows that California accounts for just over 12 percent of new and light vehicle registrations, yet that market share was able to drive change within that industry.

The NAFSB speaking as one voice could similarly set additional requirements for accreditation vendors that operate within their jurisdictions. If over 40 percent of the market had an additional requirement, it's likely easier for the private accreditation vendors to increase the requirements for everyone instead of assessing against two different standards. This can provide benefits even to jurisdictions without oversight boards or commissions.

Consensus that all eligible categories of testing should be accredited could close loopholes and prevent repeats of notable failures like the Detroit Police Laboratory. Reinstating administrative reviews and increasing the rigor of technical reviews will increase the quality of forensic providers’ work. Restoring the technical leader requirement will ensure that at least one person will be responsible for keeping abreast of the most recent advances and updates in each category of testing.

Using this mechanism to leverage change has an additional benefit. While requirements can be added to the existing ISO/IEC standards, base-level requirements of the international program cannot be removed, so quality can only improve.

The other international accreditation vendors that operate within the US should also be encouraged to offer forensic programs to build redundancy in the overall systems. While it can be challenging to get an ISO/IEC 17025 accreditation program recognized, starting a forensic program when ISO/IEC 17025 is already in place can occur comparatively quickly, especially if that program is based on a guidance document promulgated by the ILAC, such as their G19 [47].

As it turns out, one of the ISO/IEC 17025 vendors that is already a signatory to the ILAC MRA is administered by the National Institute of Standards and Technology [48]. It makes sense to foster the creation of a forensic program within the National Voluntary Laboratory Accreditation program (NVLAP). While the government can't force a private vendor such as ANAB to maintain a forensic program should their business interests change, it can require NVLAP to maintain one since it is housed within a governmental agency and maintaining it will serve a key criminal justice function.

4. Discussion

Since its inception, the lack of institutional support and a regulatory vacuum have posed significant obstacles to increasing quality in forensic practice. Over four decades ago, the field rallied to establish the first forensic accreditation programs, but agencies were slow to adopt more rigorous standards without any external requirement to do so.

The accreditation requirements related to accessing the DNA databank, and the jurisdictions that require accreditation have increased the number of forensic laboratories that have undergone the accreditation process, but both science and quality extend beyond the laboratory door. To date, no formal efforts have been made to even understand how many agencies offer forensic services in the United States.

A system of establishing infrastructure and incentives has already been proposed [49] that would establish both a mechanism and a fixed time frame for all forensic service providers (not just laboratories) to become accredited. That system relies on restructuring existing federal funding.

The proposal outlined in this perspective allows another opportunity for the forensic community to institute positive change from within. Instead of redirecting existing federal funding as the infrastructure and incentives model had done, these changes are cost-neutral and driven by existing state regulations and the competitive accreditation market.

Reminiscent of when the newly formed crime laboratory director organization created the first forensic accreditation program, the newly formed NAFSB, speaking with one voice, can substantially increase quality beyond the individual jurisdictions they serve and potentially beyond.

The reevaluation of Chevron deference has curtailed added federal regulation, but this does not prevent federal support from increasing the reliability of forensic practice. The NAFSB can encourage its members to find more forensic accreditation providers to build redundancy in the system. Still, only the federal government can ensure redundancy by requiring the NVLAP to add a forensic program to its existing ISO/IEC 17025 accreditation offerings.

Similarly, only the federal government can accurately assess the number of agencies offering forensic services through the Bureau of Justice Statistics.

Since the first Police Crime Laboratory, forensic providers have struggled with shoestring budgets, often serving their host agencies admirably, despite a lack of institutional support. However, when something goes wrong, it's not the agency's responsibility. It's ours as the forensic provider. We must remember that ultimately, we are accountable for the quality of the work we do. Even under the most challenging conditions, it is our responsibility to produce reliable results while improving the resilience of the systems in which we work.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.