Forensic bitemark identification: weak foundations, exaggerated claims

Michael J Saks; Thomas Albright; Thomas L Bohan; Barbara E Bierer; C Michael Bowers; Mary A Bush; Peter J Bush; Arturo Casadevall; Simon A Cole; M Bonner Denton; Shari Seidman Diamond; Rachel Dioso-Villa; Jules Epstein; David Faigman; Lisa Faigman; Stephen E Fienberg; Brandon L Garrett; Paul C Giannelli; Henry T Greely; Edward Imwinkelried; Allan Jamieson; Karen Kafadar; Jerome P Kassirer; Jonathan ‘Jay’ Koehler; David Korn; Jennifer Mnookin; Alan B Morrison; Erin Murphy; Nizam Peerwani; Joseph L Peterson; D Michael Risinger; George F Sensabaugh; Clifford Spiegelman; Hal Stern; William C Thompson; James L Wayman; Sandy Zabell; Ross E Zumwalt

doi:10.1093/jlb/lsw045

. 2016 Nov 23;3(3):538–575. doi: 10.1093/jlb/lsw045

Forensic bitemark identification: weak foundations, exaggerated claims

Michael J Saks ^1,^*, Thomas Albright ², Thomas L Bohan ³, Barbara E Bierer ⁴, C Michael Bowers ⁵, Mary A Bush ⁶, Peter J Bush ⁷, Arturo Casadevall ⁸, Simon A Cole ⁹, M Bonner Denton ¹⁰, Shari Seidman Diamond ¹¹, Rachel Dioso-Villa ¹², Jules Epstein ¹³, David Faigman ¹⁴, Lisa Faigman ¹⁵, Stephen E Fienberg ¹⁶, Brandon L Garrett ¹⁷, Paul C Giannelli ¹⁸, Henry T Greely ¹⁹, Edward Imwinkelried ²⁰, Allan Jamieson ²¹, Karen Kafadar ²², Jerome P Kassirer ²³, Jonathan ‘Jay’ Koehler ²⁴, David Korn ²⁵, Jennifer Mnookin ²⁶, Alan B Morrison ²⁷, Erin Murphy ²⁸, Nizam Peerwani ²⁹, Joseph L Peterson ³⁰, D Michael Risinger ³¹, George F Sensabaugh ³², Clifford Spiegelman ³³, Hal Stern ³⁴, William C Thompson ³⁵, James L Wayman ³⁶, Sandy Zabell ³⁷, Ross E Zumwalt ³⁸

¹Regents Professor, Arizona State University, Phoenix, AZ, USA.

²Professor and Conrad T. Prebys Chair, Salk Institute for Biological Studies, La Jolla, CA, USA.

³President of the Forensic Specialties Accreditation Board, CO, USA.

⁴Professor of Medicine, Harvard Medical School, Boston, MA, USA, and Brigham and Women's Hospital, Boston, MA, USA.

⁵Associate Clinical Professor, Ostrow School of Dentistry of USC, Los Angeles, CA, USA.

⁶Associate Professor, SUNY at Buffalo School of Dental Medicine, Buffalo, NY, USA.

⁷Director, South Campus Instrument Center at the State University of New York School of Dental Medicine, New York, NY, USA, and Adjunct Professor of Art Conservation, Buffalo State College, Buffalo, NY, USA.

⁸Bloomberg Distinguished Professor and chair of the Molecular Microbiology and Immunology Department, Johns Hopkins School of Public Health, Baltimore, MD, USA.

⁹Professor of Criminology, Law & Society, University of California, Irvine, USA.

¹⁰Galileo Professor of Chemistry and Professor of Geological Sciences, University of Arizona, Tucson, AZ, USA.

¹¹Howard J. Trienens Professor of Law and Professor of Psychology, Northwestern University, Chicago, IL, USA.

¹²Senior Lecturer, School of Criminology and Criminal Justice, Griffith University, Mt Gravatt, QLD, Australia.

¹³Professor of Law, Beasely School of Law, Temple University, Philadelphia, PA, USA.

¹⁴Acting Chancellor and Dean and the John F. Dagardi Distinguished Professor of Law, University of California Hastings College of the Law, San Francisco, CA, USA.

¹⁵Lecturer in Law, University of California Hastings College of the Law, San Francisco, CA, USA.

¹⁶Maurice Falk University Professor of Statistics and Social Science (emeritus), Carnegie Mellon University, Pittsburgh, PA, USA.

¹⁷Professor of Law, University of Virginia, Charlottesville, VA, USA.

¹⁸Distinguished University Professor and the Albert J. Weatherhead III & Richard W. Weatherhead Professor of Law, Case Western Reserve University, Cleveland, OH, USA.

¹⁹Deane F. and Kate Edelman Johnson Professor of Law and Professor, by courtesy, of Genetics, Stanford University, Stanford, CA, USA.

²⁰Edward L. Barrett, Jr. Professor of Law Emeritus, University of California, Davis, USA.

²¹Director, The Forensic Institute in the UK, Glasgow, UK, Professor of Forensic Sciences (visiting), Staffordshire University, Stoke on Trent, UK, and University of West Scotland (honorary), Scotland, UK.

²²Commonwealth Professor & Chair of Statistics, University of Virginia, Charlottesville, VA, USA.

²³Distinguished Professor of Medicine, Tufts University School of Medicine. Boston, MA, USA.

²⁴Beatrice Kuhn Professor of Law, Northwestern Pritzker School of Law, Chicago, IL, USA.

²⁵Consultant, Department of Pathology, Massachusetts General Hospital, Boston, MA, UK, and Professor of Pathology at Harvard Medical School, Cambridge, MA, USA.

²⁶Dean, UCLA School of Law, Los Angeles, CA, USA.

²⁷Associate Dean for Public Interest & Public Service Law, George Washington University Law School, Washington, DC, USA.

²⁸Professor of Law, New York University School of Law, New York, NY, USA.

²⁹Chief Medical Examiner, Tarrant County, Texas, Fort Worth, TX, USA.

³⁰Recently retired Professor, School of Criminal Justice and Criminalistics, California State University, Los Angeles, USA.

³¹Professor of Law, Seton Hall University School of Law, Newark, NJ, USA.

³²Professor Emeritus of Biomedical and Forensic Sciences, School of Public Health, University of California, Berkeley, CA, USA.

³³Distinguished Professor of Statistics, Texas A&M University, College Station, TX, USA.

³⁴Ted and Janice Smith Family Foundation Dean and Professor, Department of Statistics, University of California, Irvine, USA.

³⁵Department of Criminology, Law & Society, Department of Psychology and Social Behavior, and the School of Law (affiliated), University of California, Irvine, USA.

³⁶Office of Research, San José State University, CA, USA.

³⁷Professor of Mathematics and Statistics, Northwestern University, Evanston, IL, USA.

³⁸Forensic Pathologist, New Mexico Office of the Medical Investigator, Professor of Pathology at the University of New Mexico School of Medicine, Albuquerque, NM, USA.

Corresponding author. E-mail: saks@asu.edu

PMCID: PMC5570687 PMID: 28852538

Abstract

Several forensic sciences, especially of the pattern-matching kind, are increasingly seen to lack the scientific foundation needed to justify continuing admission as trial evidence. Indeed, several have been abolished in the recent past. A likely next candidate for elimination is bitemark identification. A number of DNA exonerations have occurred in recent years for individuals convicted based on erroneous bitemark identifications. Intense scientific and legal scrutiny has resulted. An important National Academies review found little scientific support for the field. The Texas Forensic Science Commission recently recommended a moratorium on the admission of bitemark expert testimony. The California Supreme Court has a case before it that could start a national dismantling of forensic odontology. This article describes the (legal) basis for the rise of bitemark identification and the (scientific) basis for its impending fall. The article explains the general logic of forensic identification, the claims of bitemark identification, and reviews relevant empirical research on bitemark identification—highlighting both the lack of research and the lack of support provided by what research does exist. The rise and possible fall of bitemark identification evidence has broader implications—highlighting the weak scientific culture of forensic science and the law's difficulty in evaluating and responding to unreliable and unscientific evidence.

Keywords: admissibility, bite mark, expert evidence, forensic science

INTRODUCTION

Forensic evidence used in criminal cases has never experienced greater legal and scientific scrutiny than it does today. Some types of forensic science expert testimony, particularly some of the pattern-matching subfields, have in recent years come to be recognized as standing on foundations so weak and making claims so exaggerated that the justification for admitting them as evidence in court has been called into serious doubt. Some of those types of forensic testimony had been used for decades without any judicial concerns being raised.

The most prominent and official pronouncement of such deficiencies was given by the National Academy of Sciences’ Committee on Identifying the Needs of the Forensic Science Community in its 2009 report.¹ That report concluded that ‘The bottom line is simple: In a number of forensic science disciplines, forensic science professionals have yet to establish either the validity of their approach or the accuracy of their conclusions…’.² ‘Much forensic evidence including, for example, bite marks and firearm and tool mark identifications is introduced in criminal trials without any meaningful scientific validation, determination of error rates, or reliability testing….’.³

Studies of wrongful convictions based on DNA exonerations have found the forensic sciences to be second only to eyewitness errors as a source of false or misleading evidence contributing to erroneous convictions.⁴ Indeed, several forensic science techniques that had for decades been welcomed into American courts are now essentially, if not entirely, dead, having been found (by scientific review committees) to lack sufficient validity to continue to be offered as evidence. The eulogy for voiceprints was given by the National Academy of Sciences in 1979,⁵ following which the FBI ceased offering such experts in support of any prosecution case in chief, and the discipline slid into decline. More recently, comparative bullet lead analysis met the same fate.⁶ And, over a continuing period, numerous ‘indicators’ of arson have been determined to lack any basis in empirical reality and have been laid to rest.⁷

The most likely candidate to next join those fields and techniques in the cemetery of terminated forensic sciences is forensic odontology—the comparison of suspected bite marks (usually found in the flesh of crime victims) and the dentition of suspects. The claim of forensic dentists has been that they can accurately associate a bite mark to the one and only set of teeth in the world that could have produced the crime scene bite mark. However, as this article will explain, no sound basis exists for believing that forensic dentists can perform such a feat. Despite the lack of empirical evidence to support its claims, to date no court in the United States has excluded such expert evidence for failing to meet the requisite legal standard for admission of expert testimony. Only in rare instances did judges even raise questions concerning the trustworthiness of such evidence, even after errors in bitemark identifications came to light.⁸ This is beginning to change. In a series of high-profile cases, including DNA exonerations, bitemark identifications have been exposed as erroneous.⁹ The Texas Commission on Forensic Science has called for a ‘moratorium’ on the use of bitemark testimony in court and is auditing old cases that had involved the use bitemark evidence.¹⁰

Had the California Supreme Court decided a recent case more broadly than it did--holding (as it did not) that such evidence generally lacks reliability and validity–it might have started a cascade of similar exclusions in other jurisdictions.¹¹ Such an outcome could be viewed as atonement for California's having launched bitemark identification into its decades-long status as an accepted forensic science despite its lack of any scientific (read: empirically tested) basis.

The section immediately below reviews the legal basis for admissibility of opinion testimony on identification by means of bite marks. Our focus then turns to the scientific deficiencies of bitemark expert evidence. The next section discusses the growing recognition of doubts about the claims of forensic odontology. The section following that explains the general logic of forensic identification. The section after that discusses the claims of bitemark identification against that background of general principles. The last major section focuses on studies assessing the accuracy of bitemark identification. Finally, we conclude by examining what the life cycle of the field of bitemark identification portends for forensic disciplines more broadly, and what lessons can be drawn for both the scientific and legal communities. Forensic scientists, researchers, lawyers, judges, and policymakers must all now grapple with the legacy of decades of unreliable forensics used in our courtrooms. In addition to auditing the misuse of science in the past, difficult challenges remain to ensure that judges adequately screen scientific evidence in criminal cases in the future. The story of the rise and fall of bitemark identification suggests the perils of path dependency in judicial review of scientific evidence and the terrible miscarriages of justice that can result when judges uncritically admit unvalidated expert accept into evidence. The lessons currently being learned will need to be remembered in the decades to come.

I. LEGAL ORIGINS OF BITEMARK OPINION ADMISSIBILITY

Before 1974, forensic dentists confined themselves to trying to identify victims of natural or human-caused disasters. Frequently, those situations provided odontologists with the complete dentition of a small, well-defined set of individuals, who needed to be distinguished from each other. The method used for trying to accomplish that was to compare the victims’ dentition against their dental records, which often included full-mouth X-rays.¹²

Until 1974, the discipline refrained from trying to identify the source of a bite mark left in skin because the differences between identifying victims of mass disasters and identifying the source of a crime scene bite mark seemed to them prohibitively daunting:

The two tasks differ in important ways. In the disaster situation, there is a finite number of candidates to identify, and full dentition often is available from the victims as well as from the dental charts. In forensic bitemark cases, the number of potential suspects is huge, the bitemarks include only a limited portion of the dentition, and flesh is a far less clear medium than having the teeth (of the disaster victim) themselves.¹³

Thus, crime scene bite marks contain only a small fraction of the information available from the full dentition of mass disaster victims, and the limited dental information that is available is neither clear (flesh is far from an ideal medium for recording bite marks) nor dependably accurate (flesh is elastic and subject to distortion at the time of and after receiving the bite).

The California case of People v. Marx (1975)¹⁴ presented what three forensic dentists, led by Gerry Vale of the UCLA School of Dentistry, thought was a justifiable exception to the rule among forensic dentists that crime scene bite marks could not be trusted to yield accurate source identifications. The Marx case involved a murder victim with an elliptical laceration on her nose. The laceration was judged to be a human bite; impressions were made of the wound and compared to a cast of the defendant's teeth. At trial, the three dentists testified that in their opinion the observable portion of the unknown teeth that made the wound were indistinguishably similar to the comparable teeth of the defendant. Vale took pains to note that in many other cases they had refused to opine on the source of crime scene bite marks (for the reasons described in the preceding paragraph). This case, they felt, was a rare exception to the general rule. The teeth that made the bite mark were highly unusual and the bite mark was exceptionally well defined and three dimensional (because nasal skin is stretched taughtly over underlying bone and cartilage, nasal tissue is firmer than the tissue of other body parts where bite marks are found, such as breasts). The witnesses characterized these bite impressions as the clearest they had ever seen, either personally or in the literature.¹⁵

The defense challenged the admissibility of the expert testimony in Marx on two major grounds. First, that it was novel and not generally accepted by the field of odontology and therefore was inadmissible under California's Kelly-Frye test. Second, that it violated the doctrine of another California case, People v. Collins (1968).¹⁶Collins had held that identification conclusions based on joint probability estimates are inadmissible when the individual probabilities of the underlying attributes are unknown (and therefore are being supplied only by speculation); when the attributes are not known to be independent of each other (and therefore the ‘product rule’ typically used to combine individual probabilities to reach a joint probability conclusion is inapplicable and produces inaccurate and exaggerated conclusions); and that when the conclusion is interpreted misleadingly to suggest a tiny (or zero) probability that someone other than the defendant could have been the perpetrator.¹⁷

The bitemark expert evidence was admitted at trial and the resulting conviction was appealed. The court of appeals turned away the first ground of attack by interpreting a technique's novelty to refer not to the novelty of the identification theory being employed, but to the tools employed to visualize the bite mark and the suspect's dentition. On that, the court opined that the experts ‘applied scientifically and professionally established techniques—X-rays, models, microscopy, photography—to the solution of a particular problem which, though novel, was well within the capability of those techniques’.¹⁸

The second ground was disposed of by emphasizing that, of the forensic dentists who testified, none was ‘engaged in a “trial by mathematics” [citing Collins] on or off the stand’. Consequently, ‘[t]here was no error’.¹⁹ As the court saw things, although the underlying logic of the witnesses’ conclusions followed precisely the steps of reasoning prohibited by Collins, because the speculative data were never made explicit to a jury (but kept implicit within the experts’ theory of identification) the opinions were protected from being excludable under Collins.²⁰

Moreover, the court thought that Kelly-Frye was inapplicable in Marx, reasoning that such a test applied only to evidence that was indecipherable without an expert's interpretation, whereas Marx involved models, X-rays, and slides of the victim's wounds and the accused's dentition, all of which were clearly visible for the jurors to view, assess, and verify on their own during court proceedings, without having to rely on the expert odontologist as a necessary intermediary.

The most sensible, and charitable, reading of Marx would be that the court understood, along with the forensic dentists, that the circumstances of the injury presented an unusually stable bite mark of an apparently very unusual set of teeth. In short, the offer and the admission in Marx constituted a rare exception to the general rule (among forensic dentists) that bite marks were a poor basis for trying to compare patterns.

Marx became the paradoxical seed from which most, if not all, subsequent decisions about admissibility of bitemark expert testimony grew. Although the experts in Marx agreed to testify only because they regarded its facts as an exceedingly rare, and therefore justifiable, exception to the field's general belief that accurate source identification was not possible using bite marks in flesh, subsequent courts ignored that distinction and cited Marx for the far more general proposition that bite marks in flesh ‘could’ be associated with their sources with a high degree of accuracy. Marx came to stand for the very proposition that the experts in the case, and their field, had up to that point explicitly, collectively rejected.

What had been an exception to the rule magically became the rule, not only for courts but for forensic dentists as well. But, ironically, rather than forensic dentists convincing courts that their field could accurately identify the sources of bite marks, the courts convinced forensic dentists that they could do what until then they doubted they could do.

The following year, Illinois considered for the first time the issue of admissibility of bitemark evidence. Relying in part on Marx, in People v. Milone (1976), the Illinois Court of Appeals held it admissible as ‘a logical extension of the accepted principle that each person's dentition is unique’.²¹ The court based this on its earlier recognition of the identification of accident victims from their dental records. The testimony of three forensic dentists was offered by the prosecution and four by the defense. The defense experts testified and cited odontological literature showing, at the least, an absence of any consensus among forensic dentists as to whether perpetrators could be identified from bites left in the flesh of victims. Notwithstanding the controversy in the trial record and in the literature, the court found that the general acceptance standard had been met. Moreover, it held that questions about the scientific soundness of the prosecution's experts’ claims went to the weight of the expert testimony, not to its admissibility, and thus were questions for the jury, not for the court.²²

By 1978, the California Court of Appeals flatly held that the testimony of three forensic odontologists established that bitemark identification had attained the required general acceptance in the relevant scientific community.²³

Daubert, despite its requirement for establishing scientific validity as a condition of admissibility, appears to have changed nothing. The two earliest post-Daubert cases, in federal courts, decided more than a decade after Daubert illustrate the difficulty courts have in focusing on the validity of the asserted forensic expertise.

In Burke v. Town of Walpole (2004),²⁴ the plaintiff alleged civil right violations against Massachusetts for his wrongful arrest and imprisonment, based heavily on a bitemark examination which purported to identify him as the person whose bite mark was found on the body of a murder victim. He was later exonerated by DNA typing. In the course of drafting recommended findings concerning the Commonwealth's motion to dismiss, the federal magistrate judge appeared never to doubt the validity of bitemark expertise though the best the court could do to support its faith was to cite cases that cite cases that express the same credulousness.

In Ege v. Yukins (2005)²⁵ in ruling on a habeas petition, the district court found the admission of bitemark expert opinion at the original trial to be so ‘unreliable and grossly misleading’, id. at 880, as to constitute a fundamental denial of due process, id. at 880. The defendant had been convicted of murder 9 years after the underlying crime took place and served more than 10 years of a life sentence by the time the federal court granted relief.

At the original trial, the defendant had been convicted in large part on the testimony of a forensic dentist whose opinion it was that a mark on the cheek of the victim, visible in a photograph of the corpse, was a human bite mark and that the mark matched the dentition of the defendant and no one other than the defendant. The odontologist stated that out of the 3.5 million people residing in the Detroit metropolitan area, the defendant was the only one whose dentition could match the asserted bite mark on the victim's cheek. The petitioner argued that the bitemark testimony had been improperly admitted because it lacked any scientific foundation and that the statistical probability given had an exaggerated impact on the jury. The court ruled that ‘there is no question that the evidence in the case was unreliable and not worthy of consideration by a jury’.²⁶ The court's conclusion could hardly be more clear. But the court's condemnation of the bitemark testimony did not go to fundamental weaknesses of bitemark comparison; it was instead aimed at case-specific, even witness-specific, problems.

One ground for the court's concern was that the comparison was made using a photograph of the wound. What was problematic about this the court does not say.²⁷ A second factor was the court's perception that this particular expert witness was singularly incompetent: ‘Dr. Warnick thoroughly has been cast into disrepute as an expert witness and several convictions based on his testimony have been undermined and overturned’.²⁸ Since those troublesome cases occurred long after the trial that was the subject of habeas review, they were nothing the trial court could have known a decade earlier. The habeas court does not say what was wrong with Dr. Warnick's examinations, or if they were standard practice among forensic dentists. The final flaw found by the court was that Dr. Warnick expressed his opinion through an explicit probability value. The court goes on at some length, discussing and citing numerous cases that raise doubts about inferences based on probability estimates. What the court failed to appreciate is that all of forensic odontology relies on these same notions to reach their conclusions of identity. That Dr. Warnick expressed his conclusion by uttering a number while his brethren typically do so by asserting verbally that dentition is unique among all humans, that the defendant's dentition matches the bite mark, and therefore the defendant has to be the source of the bite mark (to the exclusion of all other possible sources). Thus, Dr. Warnick's number was less extreme and no more scientifically unjustified than the verbal formulation typically presented by forensic dentists. The court seems unaware of that.

Ege, like Burke, assumes the general soundness of the methods of bitemark comparison, but finds fault with the particular individual performing the comparisons. By attacking this particular witness and his particular testimony with such vigor, the court avoided placing the field's more general shortcomings under scrutiny. The problem with the expert witness seems not to be that he deviated from his discipline's generally accepted practices so much as that he followed them.

As of this writing, no court we are aware of has ever excluded or otherwise held forensic bitemark expert testimony to be inadmissible. Perhaps that is because no court has thoughtfully compared the claims of bitemark identification to the (lack of) scientific foundation for those claims. They have admitted the testimony essentially because other courts admitted it. Even a radical change in the test for admission—that is, the U.S. Supreme Court's adoption of the Daubert tetralogy—did not change that practice.

A treatise on forensic scientific evidence and the law, the lead author of which had been a forensic scientist before turning legal scholar, described these developments two decades later, saying:

The wholesale acceptance, by the courts, of testimony on bitemark identification has transformed the profession. Whereas prior to 1974 the main thrust of forensic dentistry was to prove identity of persons by means of a comparison of postmortem and antemortem dental records in mass disasters, the profession has changed direction and is now heavily involved in assisting prosecutors in homicide and sex offense cases. Having received judicial approval of bitemark comparisons, there seems to be no more limit on the extent of forensic odontological conclusions.²⁹

II. GROWING DOUBTS

Beliefs about the capacity of bitemark comparisons to accurately identify the source of a questioned bite mark have followed a trajectory from widespread skepticism through widespread credulity to a growing return to doubt. That growing doubt is based on the emerging realization that the field stands on a quite limited foundation of scientific fact, that there is ‘a lack of valid evidence to support many of the assumptions and assertions made by forensic dentists during bite-mark comparisons’,³⁰ and that error rates by forensic dentists are perhaps the highest of any forensic identification specialty still being practiced.³¹ Bitemark testimony has been ‘introduced in criminal trials without any meaningful scientific validation, determination of error rates, or reliability testing…’.³²

Those realizations have been taken up most prominently in the work of a committee of the National Academy of Sciences, which reviewed the scientific support for the claims of bitemark identification, among others, and found serious deficiencies.³³ The Committee on Identifying the Needs of the Forensic Science Community was co-chaired by Judge Harry Edwards, of the U.S. Court of Appeals for the D.C. Circuit, who described the Committee's work:

[The Committee spent] more than two years… listening to testimony from and reviewing materials published by countless experts, including forensic science practitioners, heads of public and private laboratories, directors of medical examiner and coroner offices, scientists, scholars, educators, government officials, members of the legal profession, and law enforcement officials. Not only were we trying to understand how the forensic science disciplines operate, we were also trying to determine the extent to which there is any… scientific research to support the validity and reliability of existing forensic disciplines; in particular, we were looking for scientific studies that address the level of accuracy of forensic disciplines that rely on subjective assessments of matching characteristics. We invited experts in each discipline to refer us to any such research….³⁴

The Committee completed its work and issued its report in 2009. Several observations and conclusions can be drawn from the report relevant to evaluating asserted bitemark identification expertise, including the following.

Bitemark identification was seen as a field in which ‘forensic science professionals have yet to establish either the validity of their approach or the accuracy of their conclusions, and the courts have been utterly ineffective in addressing this problem’.³⁵

‘Although the majority of forensic odontologists are satisfied that bite marks can demonstrate sufficient detail for positive identification [of a perpetrator], no scientific studies support this assessment….’³⁶ ‘[T]he scientific basis is insufficient to conclude that bite mark comparisons can result in a conclusive match.’³⁷

One reason for doubts about ‘the value and scientific validity of comparing and identifying bite marks’³⁸ is the unsatisfactory nature of skin as a substrate for registration of tooth impressions: ‘Unfortunately, bite marks on the skin will change over time and can be distorted by the elasticity of the skin, the unevenness of the bite surface, and swelling and healing. These features may severely limit the validity of forensic odontology’.³⁹ This aspect of bitemark identification sets it apart from other types of forensic pattern-comparison techniques.

There is no science on the reproducibility of the different methods of analysis that lead to conclusions about the probability of a match. This includes reproducibility between experts and with the same expert over time. Even when using the guidelines, different experts provide widely differing results and a high percentage of false positive matches of bite marks using controlled comparison studies.⁴⁰

The NAS Committee recognized the work of cognitive scientists showing that, when viewing ambiguous information, the observer's mind tends to see what the observer expects or hopes to see.⁴¹ Ambiguities are resolved as being consistent with expectations, and bitemark experts do not generally employ procedures for preventing such errors:

[F]orensic odontology suffers from the potential for large bias among bite mark experts in evaluating a specific bite mark in cases in which police agencies provide the suspects for comparison and a limited number of models from which to choose from in comparing the evidence. Bite marks often are associated with highly sensationalized and prejudicial cases, and there can be a great deal of pressure on the examining expert to match a bite mark to a suspect. Blind comparisons and the use of a second expert are not widely used.⁴²

In concluding that ‘[m]ore research is needed to confirm the fundamental basis for the science of bite mark comparison’, the NAS Report summarized ‘[s]ome of the basic problems inherent in bite mark analysis and interpretation’ as follows.

The uniqueness of the human dentition has not been scientifically established.
The ability of the dentition, if unique, to transfer a unique pattern to human skin and the ability of the skin to maintain that uniqueness has not been scientifically established.
1. The ability to analyse and interpret the scope or extent of distortion of bitemark patterns on human skin has not been demonstrated.
2. The effect of distortion on different comparison techniques is not fully understood and therefore has not been quantified.
3. A standard for the type, quality, and number of individual characteristics required to indicate that a bite mark has reached a threshold of evidentiary value has not been established.⁴³

III. THE LOGIC OF FORENSIC IDENTIFICATION − −GENERALLY

Forensic identification, including bitemark identification, involves two indispensable steps.⁴⁴ The first step is to compare the crime scene markings to the possible sources of that mark.⁴⁵ The examiner compares images of the questioned markings to those from the known and makes a judgement about whether they differ to an extent that the suspect should be excluded as the source, or that the similarities seem so great that the suspect should be included in the pool of possible contributors. In the case of crime scene markings created by one object leaving markings of itself on another object—such as a fingerprint onto a surface, a firearm barrel onto a bullet, or teeth onto skin—the faithfulness of the transfer from the original to the receiving surface, and the ability of the receiving surface to retain the impression unchanged, are essential to the probativeness of the comparison of the mark on the receiving surface to a suspected source.

A. Problems with declaring a ‘match’

In comparing the images of the questioned and the known, if examiners are left to their own subjective judgement of how similar two images need to be in order to declare them similar enough to be included in the pool, then inconsistencies will occur when different examiners look at the same evidence. The less well the criteria are defined and held in common among examiners, the more rife with inconsistency their work will be.⁴⁶

The description in the preceding paragraph is careful to avoid using the term ‘match’. Though employed with decreasing frequency, that word is still in wide use and is unexpectedly troublesome. The term has multiple meanings in the forensic context, which are easily conflated. The term risks misleading factfinders into believing the expert's conclusion is more certain than pattern-matching conclusions can be.

One meaning has to do with observation. It says that the questioned and the known images share many similar features. This observation is almost never (and perhaps literally never) that the two images are identical, or indistinguishably alike. Differences are always present in all forensic pattern matching. Part of the examiner's task is to try to decide which differences can safely be disregarded as unimportant and which similarities are of significance. Here, one might say, ‘they match’—if that statement simply means that the questioned and the known are highly similar in appearance.

A second meaning has to do with inference. The examiner's ultimate goal is to try to infer whether the questioned and the known ‘share a common source’. Did the finger that made the file print make the latent print? Did the gun that fired the crime scene bullet fire the test bullet? In line with this meaning, one would like to say, ‘it's a match’—that is, the one and only source of the crime scene evidence has been identified.

Such a conclusion can never be reached in more than a probabilistic sense, and for that reason the assertion of a ‘match’ to mean a definite inference of common source is misleading. It is impossible to know how many other sources could have made marks as similar to the crime scene mark as the one under examination. The most that can justifiably be said is that the known image belongs to a pool containing an unknown number of other objects that can produce images with very similar characteristics. This is precisely why DNA typing produces ‘random match probabilities’ (RMPs) rather than assertions that ‘the’ source of the crime scene DNA has been found. The RMPs provide the best available sense of the probability that a randomly selected person's DNA would ‘match’ the crime scene DNA (in addition to that of a suspect whose DNA profile has been found to ‘match’).⁴⁷

Upon hearing an expert witness state that an assertedly scientific process has determined that the questioned and the known are ‘a match’, factfinders can be forgiven for mistakenly thinking the identification is more certain than it is capable of being.⁴⁸

A third meaning of the word ‘match’ had been used until recently by forensic dentists. The American Board of Forensic Odontology's⁴⁹ official guidelines for testifying to bitemark comparison opinions approved use of the term ‘match’ to mean: ‘Some concordance, some similarity, but no expression of specificity intended; generally similar but true for large percentage of population’.⁵⁰

Upon hearing that a suspected source and a crime scene object ‘matched’, laypersons in one study interpreted that term to indicate the strongest linkage (even though it was intended to be the weakest linkage) of any of the terms then available to forensic dentists for expressing their sense of the association between a bitemark and a suspect's dentition.⁵¹ In the current ABFO Diplomates Reference Manual (2013), the term ‘match’ has been eliminated as an acceptable term for expressing opinions about bitemark source attribution.⁵²

To avoid the misunderstandings from which the term ‘match’ suffers, this article tries to avoid its use as much as possible. When that is not possible, we try to use it carefully.

B. Evaluation of an inclusion

If the decision reached by the examination process is inclusion of the suspected source, the next step is to evaluate the meaning of that inclusion. Its probativeness depends upon how many other members of the population could also have produced markings with a very similar appearance to the crime scene marks.

This evaluation is done most transparently in the methods of DNA comparison for single-source crime stains, where sampling of the relevant population has been conducted and informs examiners about the frequency of occurrence of the alleles being compared. That information allows calculation of the RMP, that is, the probability that a random member of the population has the same DNA profile as that collected at the crime scene. The more people in the population with the same profile (the larger the RMP), the less probative is the fact of the suspected source having the same profile. The fewer people in the population who share the profile (the smaller the RMP), the more probative is the fact of the suspected source having the same profile as the crime scene DNA.

Thus, some estimates of the size of the subpopulation that shares a profile with the crime scene mark are necessary to evaluate the meaning of a ‘match’. That is not to say it must be done just as DNA typing does it. But without some methods for evaluating the meaning of a suspected source having similar appearance to the crime scene evidence, a factfinder has no way to gauge how probative that fact is, and might be misled by testimony saying only that a suspected source has been judged to ‘match’ the crime scene mark—in whatever terms that fact might be expressed.

Because the forensic identification process is fundamentally probabilistic, absolute statements of identification are insupportable. ‘[T]he scientific basis is insufficient to conclude that bite mark comparisons can result in a conclusive match.’⁵³ Thus, any opinions expressed in terms suggesting pinpoint identification—such as ‘identification to the exclusion of all others’, ‘indeed and without doubt’, ‘certainty’, and ‘perfect match’—have been properly criticized by numerous authorities as exceeding what the forensic identification process is capable of. ⁵⁴ Such extreme opinions are (now) disapproved by the ABFO as well: ‘Terms assuring unconditional identification of a perpetrator, or without doubt, are not sanctioned as a final conclusion’.⁵⁵ At the same time, in contradiction, the ABFO currently permits a conclusion that a suspect is ‘the biter’, which is an expression of unconditional identification. And, prefatory to all of the currently approved conclusions,⁵⁶ the ABFO requires: ‘All opinions stated to a reasonable degree of dental certainty’.⁵⁷

Recently, a subcommittee of the National Commission on Forensic Science has proposed that the Commission issues a caution against the use of the expression, ‘to a reasonable scientific certainty’, or its discipline-specific variants, to characterize an expert opinion: ‘It is the view of the National Commission on Forensic Science that the scientific community should not promote or promulgate the use of this terminology’. The National Commission on Forensic Science subcommittee explained that the expression has no scientific meaning and tends to be misleading to factfinders because it asserts certainty.⁵⁸

Exaggerated testimony expressing conclusions about pattern-comparison evidence—that is, testimony that exceeds what a field's knowledge and techniques can support—led the FBI to agree to review approximately 2500 cases worked from 1972 to 1999 by its own microscopic hair examiners. With about half the cases reviewed, ‘by the FBI's count examiners made statements exceeding the limits of science in about 90 percent of testimonies, including 34 death-penalty cases.’⁵⁹

IV. BITEMARK IDENTIFICATION IN LIGHT OF THE LOGIC OF FORENSIC IDENTIFICATION

Against the background of forensic identification more generally, the special difficulties of bitemark identification can be more readily appreciated.

A. The source of the bite mark

When trying to identify a decedent who has a full mouth of teeth by comparing those to dental records, a great deal of information is available.

The human adult dentition consists of 32 teeth, each with 5 anatomic surfaces. Thus, there are 160 dental surfaces that can contain identifying characteristics. Restorations, with varying shapes, sizes, and restorative materials, may offer numerous additional points of individuality. Moreover, the number of teeth, prostheses, decay, malposition, malrotation, peculiar shapes, root canal therapy, bone patterns, bite relationship, and oral pathology may also provide identifying characteristics.⁶⁰

But when trying to identify the source of a bite mark, only a fraction of that information is available:

[I]n the typical bite mark case, all 32 teeth cannot be compared; often only 4 to 8 are biting teeth that can be compared. See Fig. 2, which presents molds of the dentition from two different people (drawn from a sample of 500) whose six front teeth are indistinguishably alike. Similarly, all five anatomic surfaces are not engaged in biting; only the edges of the front teeth come into play.⁶¹

Figure 2. — Indistinguishably similar dentition. Three-dimensional models of two different people's dentitions in which the six anterior (front) teeth were found to have the same three-dimensional shape, based on measurement error determined by repeated measurement. [Reprinted with permission of creator, Peter Bush.]

Figure 1. — Bitemark evidence from trial of Arizona v Krone. [In public domain.]

Moreover, the amount of information contained in the dentition involved in creating a bite mark is far less than that contained in fingerprints, DNA, and most other forms of forensic identification. Thus, the process of bitemark identification begins with a serious disadvantage relative to other types of forensic evidence: less information from the unknown specimen with which to work.

B. The substrate onto which a bite pattern is transferred

The potentially identifying information contained in the teeth that create a bite mark has to be captured by the material (the substrate) into which the bite is impressed. If the image of the bite mark in skin is undependable and unstable, then examiners cannot know whether they are looking at a true picture of the dentition that created the bite mark, or a distorted picture.⁶²

In the crime context where bite marks are found, that substrate usually is skin. Skin is a poor substrate for recording the pattern of teeth. It is far less able than the modern dental materials used in dental offices to capture and dependably retain the features of, say, a tooth being replaced by a crown. Skin is a viscoelastic material. The elastic property means that indentations left by teeth will rebound, leaving potentially no record of the three-dimensional structure of the biting edges of teeth. This reduces the information that may be used for comparison. The analysis then might typically consist of comparison of a bruise to a dental model. Because a bruise consists of diffusion of blood from crushed capillaries, no precise measurements can be made for comparison.

To further complicate the situation, biting in the criminal context typically occurs during struggles, during which skin is stretched and contorted at the time the bite mark is created. When the skin returns to its normal shape, the resulting image of the biter's dentition can be distorted to an unknown extent. Figure 3 illustrates what can happen when a marking is placed on skin that has been stretched and the skin then returns to its normal shape. Similarly, the position in which body parts are positioned postmortem can change the shape of the bite mark. Figure 4 illustrates this problem with an actual bite mark on the skin of a human cadaver.

Figure 3. — Two identical marks on human skin. The lower mark has been distorted by applying pressure to the area (duplicating Devore's Test). [In public domain.]

Figure 4. — Changes in bitemark appearance depending upon how the body part is positioned. The bite was inflicted with the arm straight at the side (left). The bitemark is outlined in black for ease of viewing; biter's overlay is in blue. Notice the alteration to the bite pattern when the arm is positioned over the head (right). [Reprinted with permission of creator, Peter Bush.]

In addition, live flesh reacts to injury, becomes inflamed, changes shape, and swells as healing begins. After death, changes in the skin and flesh occur due to decomposition, animal predation, insect activity, embalming, and environmental factors as well as other processes.

The pliability, elasticity, and reactivity of skin and flesh all create a major challenge for bitemark identification and set it apart from other kinds of pattern-comparison forensic identification. As the NAS Report concluded in regard to these substrate problems, ‘These features may severely limit the validity of forensic odontology’.⁶³

C. Methods of comparison

When a forensic dentist undertakes to compare a questioned bite mark with a suspect's dentition, numerous techniques exist and are recognized by the ABFO Guidelines, including drawing bitemark images by hand.

The issue of the multiple methods of bitemark analysis continues to thwart any attempts to standardize procedures to any sort of ‘gold standard.’ The use of digital methods in the superimposition of bitemark evidence appears to be increasing, although the older, more experienced forensic dentists still seem to resist the use of two dimensional computer methods.⁶⁴

Although there has been some research-comparing techniques, finding some to be significantly better than others at facilitating the visualization of bitemark-to-dentition similarities and differences,⁶⁵ the guidelines do not specify criteria under which one method might be preferred to another. And, in any event, there is no oversight, so forensic dentists are free to use whichever method they happen to be familiar with or prefer.

Nor has the field of forensic odontology developed inclusion/exclusion criteria. Each examiner is left to form his or her own judgement about which features of the bite mark to compare and whether to declare a (suspected) bite mark and a suspect's dentition to be so similar that the examiner should declare an inclusion. Absent from bitemark analysis are ‘precise and objective criteria for declaring matches’, considered to be essential elements of any field of forensic identification.⁶⁶

D. Lack of data on population frequencies

To this point, we have addressed potentially insurmountable difficulties in bitemark identification that involve nothing more than the seemingly straightforward task of comparing a questioned bitemark to a suspect's dentition. Assume, however, an optimal case: sufficient information from source dentition exists and has been impressed upon a stable substrate on a victim's body; that sound methods have been employed to visualize and compare the bite mark on the victim and a suspect's dentition; that valid criteria have been developed for deciding when to include and when to exclude dentition as a possible source; and that a forensic dentist has reached a justifiable conclusion that the images were sufficiently similar to include. The next step would be to assess what that decision can tell a factfinder about the likelihood that the suspected person's dentition did in fact produce the bite mark. As discussed earlier, such an evaluation depends upon estimating the frequency of similar patterns in the relevant population.

Unfortunately, forensic dentists have very little information of the kind needed to make an informed assessment. ‘If a bite mark is compared to a dental cast using the guidelines of the ABFO, and the suspect providing the dental cast cannot be eliminated as a person who could have made the bite, there is no established science indicating what percentage of the population or subgroup of the population could also have produced the bite.’⁶⁷ Actual probabilities are not known because no population studies have been carried out to determine what features to consider, much less the actual degree of variation in teeth shapes, sizes, positions, etc., that exist in the population.⁶⁸ Work to remedy this shortcoming is at an early stage.⁶⁹

Recent studies, however, have cast light on the risk of erroneously calling similar dentitions a ‘match’ by establishing ‘match’ rates among dental populations using methods of measurement resolution that are better than can possibly be achieved with marks on skin. In these studies, a ‘match’ was defined as specimens that could not be determined as distinguishable within measurement error.⁷⁰ A fundamental conclusion from these studies was that as any database of dental arrangement increases in size, the probability of one dental arrangement matching another one increases. This was especially true in analysis of orthodontically treated dentitions, in which dental arrangements are purposely made homologous.⁷¹ The latest of these studies (n = 1099) documented the most common patterns of dental malalignment three dimensionally in a large population. This study also found that the effect of increasing distortion (reducing measurement resolution) was that dramatically larger numbers of dentitions ‘matched’.⁷² In short, these recent studies indicate that, given relatively large numbers of people with seemingly unusual misalignments of teeth, compared using the relatively poor resolution of teethmarks on skin, the risk of false positive errors is quite real.

In the absence of data concerning population frequencies of dental characteristics, how have forensic dentists assessed the value of an inclusion? One way has been to speculate or guesstimate about the population frequencies of the characteristics of biting teeth. A forensic dentist might judge a bite mark to have been made by a pattern of teeth that seems unusual in his or her experience. On occasion, a source's teeth are so unusual that they are obvious outliers; then, when a suspect's teeth are deemed closely similar (a well-defined bite mark, impressed into a stable substrate), the probability is smaller that a different person will have produced the bite mark.⁷³ Nevertheless, a forensic dentist's placing too much faith in the apparent unusualness of a source dentition has led to known erroneous convictions. There is no escaping the fact that forensic identification is an essentially probabilistic endeavor. For the great majority of bite marks, however, population frequencies will necessarily be higher than in the very unusual cases, and the risk of erroneous identification greater.⁷⁴

E. Uniqueness

The conventional solution to the problem of assessing the meaning of a ‘match’ has been to assume uniqueness. ‘Identification of a suspect by matching his or her dentition with a bite mark found on the victim of a crime rests on the theory that each person's dentition is unique’.⁷⁵ But as the uniqueness assumption has increasingly come to be recognized as unproved and unsound, it also has ceased to serve as a viable solution to the problem of how to evaluate the meaning of a high degree of similarity between a bite mark and a suspect's dentition.

Two different concepts are expressed by the notion of bitemark ‘uniqueness’.⁷⁶ One is the claim that no two dentitions duplicate one another in absolutely every respect. This has been termed ‘mere uniqueness’. An even stronger claim is being made by forensic dentistry: not only that all dentitions are unique, but also that every bite mark produced by those dentitions can be associated only with themselves and not with any other dentition. If this claim were true, it would indeed be possible to conclude that a dentition found consistent with a mark is the source of that mark. But we know from the substrate problems described, above, and from systematic empirical research as well as observations by practicing forensic dentists that repeated bites by a single set of dentition produce very different bite markings.

The advantage of adopting and asserting the assumption of uniqueness is that it obviates the need to collect, analyse, and employ information about the population distribution of dentitions and bitemark characteristics. Much of the hard work of empirical research can be dispensed with. If no two dentitions belonging to different persons can possibly produce bite marks that are indistinguishably alike or confusingly similar, then a judgement that a questioned bite mark looks much like a suspect's dentition is assumed to mean that the suspect is ‘the’ source of the bite mark, not merely a member of a pool containing some unknown number of possible contributors.

The problem with the assumption of uniqueness is that it is nothing more than ipse dixit. The NAS Report on forensic science stated:

No thorough study has been conducted of large populations to establish the uniqueness of bite marks; theoretical studies promoting the uniqueness theory include more teeth than are seen in most bite marks submitted for comparison. There is no central repository of bite marks and patterns. Most comparisons are made between the bite mark and dental casts of an individual or individuals of interest. Rarely are comparisons made between the bite mark and a number of models from other individuals in addition to those of the individual in question.⁷⁷

In sum, ‘The committee received no evidence of an existing scientific basis for identifying an individual to the exclusion of all others’.⁷⁸

A recent review sought to examine all empirical research aimed at determining whether all human dentition is unique.⁷⁹ Following an extensive bibliographic database search, 13 studies were found and each was reviewed in detail. None was able to support a conclusion of dental uniqueness. Nine of the studies explicitly failed to find uniqueness. Four claimed to have succeeded, but were found to be methodologically incapable of supporting the asserted conclusions. Four additional studies⁸⁰ found specimens in the study populations that were indistinguishable within measurement resolution—that is, their differences did not exceed the margin of error for the study population.

These findings bring the notion of dental uniqueness, central to bitemark analysis, into considerable doubt. As the assumption of uniqueness fades away, so does the claim that bitemark comparison can dependably link a bite mark to its source.

In light of these developments, the ABFO has recently backed away from the theory of uniqueness and the associated notion of identification-to-the-exclusion-of-all-others.⁸¹ The ABFO has gone so far as to suggest that any attempt to narrow identification to a single individual has to be limited to cases involving ‘closed populations’—that is, cases in which only a small number of known persons could have been in a position to inflict the questioned bite. Forensic dentists then need only distinguish among the dentition of a handful of known people, not speculate about tens of millions of unknown dentitions.⁸²

V. HOW ACCURATE ARE BITEMARK IDENTIFICATIONS?

The empirical research described in this section is noteworthy, first, for how little of it there is and, second, for how much of what does exist refutes the claims of forensic dentists regarding their ability to identify the source of a bite mark.

A. Measuring error—generally

In the context under discussion, decision error consists of two distinct types: a ‘false positive’, which is a decision that a bite mark came from a specific set of teeth when in fact it was made by other teeth and a ‘false negative’, a decision that a bite mark did not come from a specific set of teeth, when in fact it did. However, the forensic comparisons are reported—‘match’, ‘consistent with’, ‘cannot exclude’—the opinions would all be classified as false positives if the ‘ground truth’ is that the bite mark did not actually come from the teeth of the suspect.⁸³

False-negative errors could occur for many reasons—some pertaining to the circumstances of the bite and the substrate receiving the bite, some pertaining to the medium the examiner is using to visualize the questioned and known patterns (eg photographs under different lighting conditions), others pertaining to the decision-making machinery of the examiners. Careful research would need to be designed in order to isolate the various possible causes of the errors and to try to develop ways to reduce errors stemming from those causes. Similarly, false-positive errors could occur for a variety of reasons, pertaining to different aspects of the bite sources, tools for and conditions of visualizing the bite marks, or the perceptual and decision characteristics of examiners.

Although the terms ‘reliability’ and ‘validity’ often are used interchangeably by laypersons, it is useful to maintain the distinction used by scientists and statisticians.⁸⁴ Scientists and statisticians distinguish between and separately measure reliability and validity. ‘Reliability’ is the extent to which a measuring instrument (including human examiners) produces the same results again and again when it measures the same thing repeatedly. Intraexaminer (or within-examiner) unreliability refers to the same examiner giving different answers on different occasions when examining the very same evidence. Interexaminer (or between-examiner) unreliability refers to different examiners examining the same evidence and reaching different conclusions about it.

Reliability concerns only consistency of measurement. It does not address whether a measurement is correct. ‘Validity’ is concerned with the question of whether a measuring instrument (including the judgements, decisions, and opinions of humans) is generating correct answers. Five forensic dentists might all agree on whether or not a suspect's dentition made a bite mark (high reliability), but they might all be incorrect (low validity).⁸⁵

B. Recent research on reliability

The ABFO recently sponsored and conducted a reliability study of the judgements of experienced, board-certified forensic dentists making very basic decisions about bite marks.⁸⁶ The researchers selected 100 photographs of suspect bitemark injuries from actual cases. These were examined by 38 ABFO-certified forensic odontologists having an average of 20 years’ experience in bitemark identification.

The 38 examiners were asked to review the injuries in each of the 100 photographs and respond to three very basic questions. As will become apparent, the greater the degree of agreement among the examiners, the more reliability is indicated (that is, repeatability of judgements by different examiners), and the lower the rate of agreement, the less reliable their judgements are. No one can know which answers were right or wrong (that is, this was not a test of validity). We can know only the extent to which they agreed or disagreed with each other.

Question 1: Is there sufficient evidence in the presented materials to render an opinion on whether the patterned injury is a human bite mark? Findings: for only 4 of the 100 cases, did all examiners agree on whether an opinion could be reached on whether an injury was a bite mark or not. For half of the cases, there was less than 71 per cent agreement. For one quarter of the cases, there was less than 47 per cent agreement.

Question 2. Is it a human bite mark, not a human bite mark, or suggestive of a human bite mark? Findings: in about a quarter of the cases, fewer than half of the examiners agreed on whether the injury was or was not a bite mark. In 71 of the 100 cases, fewer than 70 per cent agreed on whether the injury was a bite mark.

Question 3. Does the bite mark have distinct, identifiable arches and individual tooth marks?

By the time they reached Question 3, the examiners were already widely divided from each other in their opinions. Those who did not think that the injury photograph contained enough information to make a decision did not opine on whether it was or was not a bite mark. Those who did not think that the injury was a human bite mark would not be addressing whether individual tooth marks were identifiable.

Taking all three questions together, for just under half of the cases, half or fewer of the examiners agreed on the same trio of responses. For only 14 of the 100 cases, did at least 80 per cent of the examiners agree on the trio of responses.

Although no one knows which answers of which examiners were correct or not (the validity question), one can be sure that many answers were incorrect since contradictory answers cannot all be correct. The reliability of a measuring instrument sets an upper limit on its possible validity.

The study just described suggests that on this earliest threshold issue—before any of the other difficulties of bitemark ‘comparison’ have to be confronted—bitemark analysis has not been shown to be reliable (let alone valid). Put simply, if dental examiners cannot agree on whether or not there is enough information in an injury to determine whether it is a bite mark, and cannot agree on whether or not a wound is a bite mark, then there is nothing more they can be relied upon to say. Unless and until they can do this threshold task dependably, there is no other aspect of bitemark identification that can be counted upon to produce dependable conclusions.

C. Studies of forensic dentists’ accuracy in simulated bitemark lineups

Over the approximately four decades in which forensic dentists have been testifying in courts claiming the ability to accurately identify the individuals who were the sources of bite marks, remarkably few tests have been carried out to assess their accuracy. While there have been hundreds of studies of eyewitness accuracy, and many dozens of proficiency tests of forensic examiners in other fields, forensic dentists have been tested only a handful of times.

Such tests as exist present practitioners with bite marks to compare under circumstances where those conducting the study know which answers are correct and which are incorrect.

The earliest of these tests were conducted in the mid-1970s by forensic dentist David Whittaker.⁸⁷ Exemplar bites were made on pigskin. Note that pigskin is a more stable material for recording and retaining a bite mark than living human skin, so that tests using pigskin as the substrate would likely overstate the accuracy obtained by bitemark examiners. Incorrect identifications of the bites made in the Whittaker study ranged from 24 per cent under ideal conditions to 91 per cent when identifications were made from photographs taken 24 hours after the bites were made (which is more typical of how bitemark comparisons are done). Whittaker commented that, ‘the inability of examiners to correctly identify bitemarks in skin … under ideal laboratory conditions and when examined immediately after biting suggests that under sometimes adverse conditions found in an actual forensic investigation it is unlikely that a greater degree of accuracy will be achieved’.

The ABFO conducted several ‘workshops’ in which forensic dentists could test their identification skills. Only the 1999 workshop results have been made public. In that test, ‘All 95 board certified diplomates of the American Board of Forensic Odontology were eligible to participate in the study. Of the 60 diplomates who requested and were sent the study material, 26 returned the necessary data by the deadline [six months after receiving the test materials] and were included in the data results’.⁸⁸

All four of the ‘questioned’ bites were made by biters whose identity was known. Three consisted of materials from actual cases (in which the biter's identity was established by independent means), and the fourth was a bite into cheese. Each of those bite marks was compared to what in effect was a lineup of seven bites. Overall, examiners were in error on nearly half of their responses, more of those being false-positive errors (identifying a non-biter as being the biter) than false negatives (failing to identify the actual biter).⁸⁹

In 2001, in the course of evaluating digital overlays as a technique for comparing known and questioned bite marks, forensic dentists Iain Pretty and David Sweet observed levels of error by examiners that troubled them: ‘While the overall effectiveness of overlays has been established, the variation in individual performance of odontologists is of concern’.⁹⁰ Using board-certified forensic dentists to evaluate the test bite marks (made in pigskin), the study found that intraexaminer agreement (agreement with one's own prior judgements given three months earlier) ranged as low as 65 per cent. False-positive responses (affirmatively linking a bite to a person who had not made the bite) averaged 15.9 per cent (and ran as high as 45.5 per cent), while false negatives (failing to link a bite to the person who actually made it) averaged 25.0 per cent (and ran as high as 71.4 per cent).

Blackwell and colleagues in 2007 examined forensic dentists’ analyses of bite marks using 3D imaging and quantitative comparisons between human dentitions and simulated bite marks, with the bite marks recorded in acrylic dental wax—a far better substrate for bitemark comparisons than human skin—and false-positive error rates still ran as high as 15 per cent.⁹¹

D. Studies of bite marks in a cadaver model

Another line of simulation research sought to understand the ‘accuracy’ of skin as a substrate for recording bite marks. Mary and Peter Bush of the School of Dental Medicine at the State University of New York at Buffalo, along with statistician David Sheets, have produced an extensive body of research.⁹² They obtained access to a reliable supply of fresh cadavers. They designed a biting machine to inflict bites that could be fitted with various cast dentitions from their reference collection, and proceeded to apply multiple bites from the same and different dentitions to different areas of cadaveric skin. They then analysed the resulting bite marks and compared them to the dentitions in their collection, using digitized modeling and various statistical techniques.

The first major finding was that, due to the anisotropic⁹³ properties of skin, no two bite marks inflicted by the same dentition appeared the same.⁹⁴ If bite marks are not reproducible, then doubt increases about the evidentiary reliability of bitemark analysis. Both the biomechanical properties of human skin and the way it reacts to biting result in marks that often can be seen and characterized as fitting multiple different sets of dentition even within the researchers’ rather small reference sample (measured in the hundreds). The apparently ‘matching’ dentitions frequently did not include the dentition that actually did the biting, and the actually ‘matching’ dentitions frequently were not similar to each other.

These findings suggest that accurate source attributions (that is, determining which dentition made which bite), is likely to require the bites to have been in more stable substrates (such as wax or cheese). The degree of distortion found in the marks on skin was such that even large variations in tooth arrangements did not faithfully transfer, making profiling (prediction of dental characteristics) unreliable. In addition, the level of distortion was often far above the measurement resolution of dental shapes (discussed above), allowing a potential ‘match’ of numerous dentitions in any given population.

To better understand the implications of this line of work, it is helpful to keep in mind the range of possible substrates. At one extreme is the kind of material used in dental offices to create molds of patients’ dentition. That material is designed to receive and hold impressions of teeth with a high degree of accuracy and stability. There is nothing better for the purpose. At the other extreme are elastic and unstable substances that cannot capture details and that subsequently change shape, distorting the tooth impression as they do. Skin, as a substrate, is closer to the latter extreme. The research described above used cadavers. Because the skin of cadavers lacks the vital response, and does not undergo the changes caused by inflammatory reactions—while most bite marks encountered by courts have been imposed on living victims—it is important to appreciate that the substrate used in the research is more stable, closer to the dental office material end of the spectrum than living flesh is. Consequently, the research is more conservative in that by employing a more stable substrate it obtained ‘more accurate’ impressions than can be found in criminally inflicted bites. Moreover, it did so under more controlled conditions, preventing the distortion and slippage due to movement that occurs in a criminal struggle. Put simply, if the research found worrisome levels of variability in bite marks and erroneous ‘matches’, then bites from actual criminal cases will suffer from more extreme imperfections and be that much more prone to error.

CONCLUSION

The scientific community, and society generally, expects that before being offered to courts, and before courts grant broad and unqualified admission, the claims for a field's techniques will have been validated.⁹⁵ This validation has not happened for bitemark identification. Moreover, recent reviews of the field's claims, as well as recent empirical findings, have underscored the lack of reliability and validity of the most fundamental claims about the ability of forensic dentists to identify the source of bite marks on human skin. A committee of the National Academy of Sciences concluded that bitemark identification testimony has been ‘introduced in criminal trials without any meaningful scientific validation, determination of error rates, or reliability testing…’.⁹⁶ Two leading forensic dental researchers have noted that there is ‘a lack of valid evidence to support many of the assumptions and assertions made by forensic dentists during bite-mark comparisons’.⁹⁷

The claims of forensic dentistry have for decades outrun empirical testing of those claims. Rather than confirming the field's claims, recent research, described in this article, has confirmed that the foundations of bitemark identification are unsound. Asserted bitemark experts ‘have yet to establish either the validity of their approach or the accuracy of their conclusions, and the courts have been utterly ineffective in addressing this problem’.⁹⁸

The rise and coming fall of bitemark evidence has left a trail of miscarriages of justice in its path. A series of individuals have been exonerated by DNA testing in cases involving bitemark evidence and still more have been exonerated by non-DNA evidence. Some of those individuals spent years or even decades in prison. The trial judges who uncritically accepted that bitemark evidence, and the appellate judges and federal habeas judges who did the same, have now had their own judgment called into question. The opinions that rubberstamped the use of such flimsy evidence now stand as a warning to future judges that they must actually endeavor to carefully apply the law's gatekeeping criteria in criminal cases, and not simply grandfather in the evidence by citing to old opinions that themselves did not apply meaningful scrutiny.

If evidence as unreliable as bitemark evidence could go unquestioned in the courts and unsupported by research from the scientific community, what does that say about the larger field of forensics? Clearly, far more work needs to be done to improve judicial review and scientific research. It has taken more than three decades to begin to undo the massively unsupported field of bitemark evidence. Other fields, such as voiceprint identification and comparative bullet-lead analysis, did rise and fall more quickly. A wide range of forensic disciplines, however, continue to be used, despite questions about their validity. The FBI and a series of crime labs have only recently begun to examine old cases involving, for example, the use of microscopic hair comparisons.⁹⁹ Errors in calculation of DNA statistics in recent years are only beginning to be addressed by crime labs through audits.¹⁰⁰ The long tail of unsound science in the case of bitemark evidence suggests that: (i) the scientific community must more carefully engage with the research foundations of forensics, and not just in landmark but infrequent national commissions; (ii) lawyers must aggressively brief challenges to foundations of forensic techniques; and (iii) judges must be far more willing to carefully examine forensic evidence before admitting it. Many observers, including the National Academy of Sciences in its report, have called for a systemic renewal of such legal and scientific efforts and progress has been slow. The rise and impending fall of bitemark evidence powerfully illustrates the costs of the failure to assure that what enters our criminal courts is sound science.

APPENDIX

Michael J. Saks, Ph.D. (experimental social psychology), M.S.L., is Regents Professor at the Arizona State University, Phoenix, AZ, USA. He is on the faculties of the Sandra Day O'Connor College of Law and the Department of Psychology, and is a fellow in the Center for Law, Science, and Innovation. He also is an affiliated faculty member at the University of Haifa, Israel. He has taught courses in scientific evidence to appellate judges in the University of Virginia Law School's LL.M. program and Duke's “Judging Science” program, as well as to law faculty at Georgetown University and the Ohio State University. His research interests include forensic science and the law. Saks's work is among the most-cited research in the NRC report, Strengthening Forensic Science in the United States. Among over 230 other publications, he has been co-editor/co-author of Modern Scientific Evidence (five volumes), the Annotated Reference Manual on Scientific Evidence, Second, and The Psychological Foundations of Evidence Law. His work has earned a number of awards and has been cited in various judicial opinions, including several by the United States Supreme Court. His doctorate is from the Ohio State University and his M.S.L. is from Yale Law School.

Thomas D. Albright, Ph.D. (psychology and neuroscience), is Professor and Conrad T. Prebys Chair at the Salk Institute for Biological Studies in La Jolla, CA, USA. His laboratory seeks to understand the brain bases of visual perception, memory and visually-guided behavior. He is a member of the National Academy of Sciences, a fellow of the American Academy of Arts and Sciences, a fellow of the American Association for the Advancement of Science, and an associate of the Neuroscience Research Program. He served as co-chair of the National Academy of Sciences Committee on Scientific Approaches to Eyewitness Identification, which produced the 2014 report, Identifying the Culprit: Assessing Eyewitness Identification. He is a member of the National Academy of Sciences Committee on Science, Technology, and Law, and serves on the National Commission on Forensic Science. His doctorate is from Princeton University.

Thomas L. Bohan, Ph.D. (physics), J.D., President of the American Academy of Forensic Sciences, 2009–2010, and currently President of the Forensic Specialties Accreditation Board. His Ph.D. is from the University of Illinois-Urbana/Champagne and his law degree is from the University of New Hampshire School of Law. He has authored books and peer-reviewed papers in the scientific and legal professional literature. Reflecting his interest in forensic science and its admission into evidence, these publications include early commentary on the Daubert decision and an extensive review of the 2009 National Academy of Sciences report Strengthening Forensic Science in the United States. He resides on Peaks Island, ME, USA.

Barbara E. Bierer, M.D., is Professor of Medicine at Harvard Medical School and the Brigham and Women's Hospital, Boston, MA, USA. She is the Director of the Regulatory Foundations, Ethics and Law Program of the Harvard Catalyst, the Harvard Clinical and Translational center. She directs the Multi-Regional Clinical Trials Center of the BWH and Harvard, a University-wide effort to improve standards for the planning and conduct of international clinical trials. She has served as Senior Vice President, Research, at the BWH. Bierer also served as the Chair of the Secretary's Advisory Committee for Human Research Protections, DHHS, and is currently a member of the National Academies of Sciences Committee on Science, Technology and the Law. She has authored or co-authored over 190 publications. Her B.S. is from Yale University and M.D. from Harvard Medical School.

C. Michael Bowers, D.D.S., J.D., is a practicing dentist and an Associate Clinical Professor at the Ostrow School of Dentistry of USC, Los Angeles, CA, USA. Over many years he has collaborated with notable legal and forensic dental colleagues to improve the methods and results in forensic identification. His accompanying intent has been to inform the criminal justice system about bitemark identifiers' scientifically unsubstantiated and dangerous claims of certainty and reliability. Some of his empirical studies and reporting in published peer reviewed books and articles on this subject were cited in the 2009 NAS report as a partial basis for its bitemark findings.

Mary A. Bush, D.D.S., is Associate Professor at SUNY at Buffalo School of Dental Medicine, Buffalo, NY, USA. She is past president of the American Society of Forensic Odontology, is a fellow of the American Academy of Forensic Sciences, and is director of the Laboratory for Forensic Odontology Research, University at Buffalo. She is on the editorial board of the Journal of Forensic Sciences, has published numerous articles, has contributed to various textbooks, and lectures widely on the topic of forensic odontology including an invited presentation at a congressional hearing. She serves on NIST's OSAC Odontology Subcommittee.

Peter J. Bush is Director of the South Campus Instrument Center at the State University of New York School of Dental Medicine and Adjunct Professor of Art Conservation at Buffalo State College, Buffalo, NY, USA. He is a co-founder of the Laboratory for Forensic Odontology Research and a Fellow of the American Academy of Forensic Sciences. He has worked in many scientific areas, including Forensic Odontology. He has published over 60 articles and his work is referenced in numerous sources including the NASA website.

Arturo Casadevall, M.D., Ph.D. (biochemistry) is the Bloomberg Distinguished Professor and chair of the Molecular Microbiology and Immunology Department at Johns Hopkins School of Public Health, Baltimore, MD, USA. He received his graduate degrees from New York University. Subsequently, he completed internship and residency in internal medicine at Bellevue Hospital. He has authored over 630 scientific papers. He served on the National Academy panel that carried out the Review of the Scientific Approaches Used during the FBI's Investigation of the 2001 Anthrax Letters. He was elected to membership in the American Society for Clinical Investigation, the American Academy of Physicians, the American Academy of Microbiology and the National Academy of Medicine. He also serves on the National Commission on Forensic Science.

Simon A. Cole, Ph.D. (science and technology studies), is Professor of Criminology, Law & Society at the University of California, Irvine, USA. He is the author of Suspect Identities: A History of Fingerprinting and Criminal Identification (2001), which was awarded the 2003 Rachel Carson Prize by the Society for Social Studies of Science; Truth Machine: The Contentious History of DNA Fingerprinting (2008), with Michael Lynch, Ruth McNally & Kathleen Jordan; and more than 20 scholarly articles and book chapters about forensic evidence. He is co-editor of the journal, Theoretical Criminology. He received his doctorate from Cornell University.

M. Bonner Denton, Ph.D. (chemistry), is a Galileo Professor of Chemistry and Professor of Geological Sciences at the University of Arizona, Tucson, AZ, USA. He is recognized as a world leader in scientific optical imaging and development of new analytical instrumentation. His work has been recognized through numerous awards and today he is a Fellow of the American Association for the Advancement of Science, the Royal Society of Chemistry; the American Chemical Society, and the Society for Applied Spectroscopy. He served as co-author of the National Research Council Report, Strengthening Forensic Science in the United States, and is a Member of the National Commission on Forensic Science. He received his doctorate in 1972 from the University of Illinois.

Shari Seidman Diamond, J.D., Ph.D. (psychology), is the Howard J. Trienens Professor of Law and Professor of Psychology at Northwestern University, Chicago, IL, USA. At Northwestern she directs the JD/PhD program. She is also a research professor at the American Bar Foundation. She has published more than a hundred articles on legal decision-making in law reviews and behavioral science journals. She was elected to the American Academy of Arts and Sciences. She has been on advisory boards of the National Science Foundation, National Academy of Sciences (Panel on the Evaluation of Forensic DNA Evidence), National Center for State Courts, Federal Judicial Center, American Bar Association, and American Judicature Society. Her publications have been cited by federal and state courts, including the U.S. Supreme Court.

Rachel Dioso-Villa, Ph.D. (criminology, law & society), is a Senior Lecturer in the School of Criminology and Criminal Justice at Griffith University, Mt Gravatt, QLD, Australia. Her research investigates the admissibility of the forensic sciences, the validation of forensic science techniques, specifically fire investigation expertise, and the causes and correlates of wrongful conviction. Her work has appeared in the Stanford Law Review, Canadian Journal of Criminology, Law and Policy, Law Probability and Risk and the Wall Street Journal. She has received grants and fellowships from the Social Science and Humanities Research Council of Canada, the American Society of Criminology, the Canadian Foundation of University Women, and the Queensland Government.

Jules Epstein, JD., is Professor of Law at the Beasely School of Law, Temple University, Philadelphia, PA, USA. He teaches Evidence, Criminal Procedure and Criminal Law. He is faculty for the National Judicial College, teaching Evidence and Capital Case courses. Professor Epstein has worked on two DNA workgroups for NIJ, and on a working group on latent print issues for the National Institute for Standards and Technology. He is co-editor of Scientific Evidence Review: Admissibility and the Use of Expert Evidence in the Courtroom, Monograph No. 9 (ABA, 2013) and The Future of Evidence (ABA, 2011) and served as section editor for the Encyclopedia of Forensic Sciences, 2nd Edition (2013). He has lectured on forensics to judges and attorneys.

David Faigman, J.D., M.A. (psychology), is Acting Chancellor and Dean and the John F. Dagardi Distinguished Professor of Law at the University of California Hastings College of the Law, San Francisco, CA, USA. His graduate degrees are from the University of Virginia. He writes in the areas of science and the law, and constitutional law. He has published numerous books and articles concerning the use, or failure to use, scientific research in legal decision-making. His books include: Constitutional Fictions: A Unified Theory of Constitutional Facts, Laboratory of Justice: The Supreme Court's 200-Year Struggle to Integrate Science and the Law, and Legal Alchemy: The Use and Misuse of Science in the Law. In addition, he is a co-author of the five-volume treatise, Modern Scientific Evidence: The Law and Science of Expert Testimony. He served on the National Academies of Science panel investigating the scientific validity of the polygraph.

Lisa Faigman, J.D., is a Lecturer in Law at the University of California Hastings College of the Law, San Francisco, CA, USA. Her areas of expertise include scientific evidence and expert testimony, the integration of science and statistics with law and public policy, forensic evidence, and individual and public health decision-making. She has a special interest in women's health, neuroscience, and aging.

Stephen E. Fienberg, Ph.D. (statistics), is Maurice Falk University Professor of Statistics and Social Science (emeritus) at Carnegie Mellon University, Pittsburgh, PA, USA. He is a member of the Department of Statistics, Machine Learning Department, Heinz College, and Center for Statistics and Applications in Forensic Evidence. He is the author or editor of over 25 books and 500 papers and related publications, several of which deal with forensic statistics topics. He is a member of the National Academy of Sciences, and a fellow of the Royal Society of Canada, the American Academy of Arts and Sciences, and the American Academy of Political and Social Science. In January 2014 he was appointed as a member of the National Commission on Forensic Science.

Brandon L. Garrett, J.D., is Professor of Law at the University of Virginia, Charlottesville, VA, USA. His research and teaching interests include criminal procedure, wrongful convictions, habeas corpus, corporate crime, scientific evidence, and constitutional law. His recent research includes studies of DNA exonerations and organizational prosecutions. He is the author of a book examining corporate prosecutions, Too Big to Jail: How Prosecutors Compromise with Corporations (2014) and of Convicting the Innocent: Where Criminal Prosecutions Go Wrong (2011), examining the cases of the first 250 people to be exonerated by DNA testing. He attended Columbia Law School, where he was an articles editor of the Columbia Law Review, clerked for the Hon. Pierre N. Leval of the U.S. Court of Appeals for the Second Circuit, and then worked as an associate at Neufeld, Scheck & Brustin LLP in New York City.

Paul C. Giannelli, J.D., LL.M., M.S. (forensic science), is Distinguished University Professor and the Albert J. Weatherhead III & Richard W. Weatherhead Professor of Law at Case Western Reserve University. He received his J.D. degree from the University of Virginia, where he served as Articles Editor of the Virginia Law Review. His other degrees include an LL.M. from the University of Virginia, an M.S. in Forensic Science from George Washington University, and a B.A. summa cum laude from Providence College. He has authored or co-authored twelve books, including Scientific Evidence (5th ed. 2012), and has written over 200 articles, book chapters, reports, book reviews, and columns, including articles in the Columbia, Virginia, Cornell, Vanderbilt, Illinois, Fordham, North Carolina, Wisconsin, Ohio State, and Hastings law reviews. Other articles have been published in specialty journals at Northwestern, Georgetown, Texas, and N.Y.U. He is also co-author of a chapter on forensic science in the Federal Judicial Center/National Academy of Sciences, Reference Manual on Scientific Evidence (3d ed. 2011). His work has been cited in nearly 700 judicial opinions throughout this country (including seven decisions of the U.S. Supreme Court), as well as in foreign courts. In addition, he has testified before the U.S. Senate Judiciary Committee and served as: Commissioner, National Commission on Forensic Science, Reporter for the American Bar Association Criminal Justice Standards on DNA Evidence; co-chair of the ABA Ad Hoc Committee on Innocence; and a member, National Academy of Sciences, Bullet Lead Elemental Composition Comparison Committee.

Henry T. Greely, J.D., is Deane F. and Kate Edelman Johnson Professor of Law and Professor, by courtesy, of Genetics at Stanford University, Stanford, CA, USA. He specializes in ethical, legal, and social issues arising from the biosciences. He chairs the California Advisory Committee on Human Stem Cell Research and directs the Stanford Center for Law and the Biosciences and the Stanford Program in Neuroscience and Society. He is a member of the Committee on Science, Technology, and Law of the National Academy of Sciences and the Institute of Medicine's Neuroscience Forum. In 2007 he was elected a fellow of the American Association for the Advancement of Science.

Edward Imwinkelried, J.D., is the Edward L. Barrett, Jr. Professor of Law Emeritus at the University of California, Davis, USA. He is the coauthor of Scientific Evidence (5th ed. 2012) and “Reference Guide on Forensic Identification Expertise" in the Reference Manual on Scientific Evidence (3d ed. 2011). He was a member of the National Institute of Standards and Technology expert working group that released Latent Print Examination and Human Factors: Improving the Practice Through a Systems Approach (2012). He served as the legal consultant to the Surgeon General's Commission on Urinalysis Testing in the Armed Forces. He is a contributing editor on scientific evidence to Criminal Law Bulletin and was formerly the expert testimony columnist for National Law Journal.

Allan Jamieson, Ph.D. (forensic science), is Director of The Forensic Institute in the UK. He is a Professor of Forensic Sciences (visiting) at Staffordshire University and at the University of West Scotland (honorary). He is Editor-in-Chief of Wiley's Encyclopaedia of Forensic Sciences and A Guide to Forensic DNA Profiling, and has published in peer-reviewed and other journals. He was external examiner for forensic sciences at Edinburgh University and the University of Kent at Canterbury; Visiting Professor of Forensic Biology at Napier University, Edinburgh, head of Lothian & Borders Police Forensic Science laboratory, a director of Forensic Alliance, chair of the United Kingdom Forensic Toxicology Forum, chair of the Standards Committee and the Academic and Education Committee of the Forensic Science Society, and a member of the editorial board of Clarke's Analysis of Drugs & Poisons. He has testified and/or otherwise been involved in thousands of criminal cases in Scotland, Northern Ireland, England and Wales, the U.S., Australia, New Zealand, and Cyprus. His doctorate is from Strathclyde University.

Karen Kafadar, Ph.D. (statistics), is Commonwealth Professor & Chair of Statistics at the University of Virginia, Charlottesville, VA, USA. She received her Ph.D. from Princeton University, and previously held positions at NIST, Hewlett Packard, NCI, University of Colorado-Denver, and Indiana University. Her research focuses on robust methods, exploratory data analysis, and characterization of uncertainty in the physical, chemical, biological, and engineering sciences. She has been editor of several journals including, currently, biology & genetics editor for the Annals for Applied Statistics. She has served on several National Academy of Sciences committees, including those that led to the reports, Weighing Bullet Lead Evidence (2004), Strengthening Forensic Science in the United States (2009), Evaluating Testing, Costs, and Benefits of Advanced Spectroscopic Portals (2011), and Identifying the Culprit: Assessing Eyewitness Identification (2014). She is an elected Fellow of American Statistical Association, International Statistical Institute, and AAAS; a National Associate of the National Academy of Sciences; and a member of the Forensic Science Standards Board.

Jerome P. Kassirer, M.D., is Distinguished Professor of Medicine at Tufts University School of Medicine. Boston, MA, USA. He was editor-in-chief of the New England Journal of Medicine between 1991–1999. He has studied the process of diagnosis for 37 years and is author of numerous scientific papers and review articles on diagnostic reasoning and diagnostic testing and is co-author of Learning Clinical Reasoning (2010). He is co-editor of the most recent edition of the Reference Manual on Scientific Evidence (2011), the data source for federal judges, and has published concerning how information is assessed by the courts. He teaches diagnosis weekly at Tufts Medical Center in Boston and monthly at Stanford University.

Jonathan “Jay” Koehler, Ph.D. (behavioral sciences), is the Beatrice Kuhn Professor of Law at Northwestern Pritzker School of Law, Chicago, IL, USA. His Ph.D. is from the University of Chicago. He conducts research in how people reason with forensic and quantitative evidence in legal cases. He teaches classes in statistics and probability, forensic science, decision making, and evidence. He has published dozens of peer-reviewed journal articles, and is an editor of Law, Probability & Risk, and a consulting editor of Judgment and Decision Making.

David Korn, M.D., is Consultant in the Department of Pathology at the Massachusetts General Hospital and Professor of Pathology at Harvard Medical School, Cambridge, MA, USA. Previously, he was Vice-President and Dean of Medicine, and Professor and Founding Chair of Pathology, at Stanford University. For 11 years he served as Senior Vice President and Chief Scientific Officer of the Association of American Medical Colleges (AAMC), and then Inaugural Vice-Provost for Research at Harvard University. He is a member of the Institute of Medicine of the National Academies of Science (NAS). He was a founding member and served as co-chair of the NAS Committee on Science, Technology and Law, which initiated and oversaw the Reports, Strengthening Forensic Science in the United States, and Review of the Scientific Approaches Used During the FBI's Investigation of The 2001 Anthrax Letters.

Jennifer L. Mnookin, J.D., Ph.D. (history and social study of science and technology), is Dean at the UCLA School of Law, Los Angeles, CA, USA, where she also serves as David G. Price and Dallas P. Price Professor of Law, and Faculty Director of the Program on Understanding Law, Science and Evidence. Her scholarship and teaching focus on evidence, especially expert evidence and issues in forensic science. She is a co-author of two evidence treatises, The New Wigmore: Expert Evidence, and Modern Scientific Evidence (where her editorial responsibilities include the chapter on bitemark identification), and she has written numerous academic articles focusing on a variety of forensic identification disciplines, among other topics. She is currently a member of the National Academy of Science's Committee on Science, Technology and Law. Her J.D. is from Yale and Ph.D. from M.I.T.

Alan B. Morrison, J.D., is the Associate Dean for Public Interest & Public Service Law at the George Washington University Law School, Washington, DC, USA. He served for fifteen years as a member of the Committee on Science, Technology & Law of the National Academy of Sciences, which sponsored the report, Strengthening Forensic Science in the United States.

Erin Murphy, J.D., is Professor of Law, New York University School of Law, New York, NY, USA. Murphy's research focuses on forensic evidence and the use of new technologies in the criminal justice system. Her recent book, Inside the Cell: The Dark Side of Forensic DNA addresses the scientific, legal, and ethical questions raised by forensic DNA testing methods.

Nizam Peerwani, M.D., is the chief medical examiner for Tarrant County, Texas, Fort Worth, TX, USA. He also chairs the Texas Forensic Science Commission. His work has included the evaluation of genocide and human rights violations in Rwanda and Bosnia-Herzegovina. He was honored by Physicians for Human Rights for his human rights work. He is a graduate of the American University of Beirut (MD '76). He completed his residency in pathology at Baylor University Medical Center in Dallas, and is board certified in clinical, anatomic and forensic pathology.

Joseph L. Peterson, D.Crim. recently retired as Professor in the School of Criminal Justice and Criminalistics at California State University, Los Angeles, USA. His research has focused on the uses and effects of scientific evidence at key decision points in the judicial process (arrest, charging, adjudication, and sentencing). His work has also explored the quality of crime laboratory results via proficiency testing of examiners, problems due to the placement of crime laboratories within law enforcement agencies, and ethical dilemmas faced by forensic scientists practicing in an adversarial justice system. His censuses of crime laboratories for the Bureau of Justice Statistics have documented high caseloads, lengthy backlogs, and severe budgetary and personnel needs. He recently completed two National Institute of Justice studies examining the role and impact of scientific evidence in the criminal justice process, and the effects of DNA test results on sexual assault kits backlogged in Los Angeles, CA. He received the Distinguished Fellow Award from the American Academy of Forensic Sciences in 2008.

D. Michael Risinger, J.D., is Professor of Law at Seton Hall University School of Law, Newark, NJ, USA. He is a graduate of Yale College and Harvard Law School. He is a life member of the American Law Institute, and a past chair of the Association of American Law Schools’ Evidence Section. He was for 25 years a member of the New Jersey Supreme Court Committee on Evidence, and is currently a member of the Human Factors Subcommittee of the National Commission on Forensic Science. He is the author of two chapters in West's Modern Scientific Evidence (“Handwriting Identification” and “A Proposed Taxonomy of Expertise”), and also of articles on a range of subjects, including many articles on expert evidence issues, and on the convicted innocent.

George F. Sensabaugh, Jr., D.Crim., is Professor Emeritus of Biomedical and Forensic Sciences in the School of Public Health at the University of California, Berkeley, CA, USA. He also teaches at UC Davis where he is a member of the Graduate Group in Forensic Science. His research interests include the application of the biosciences in forensic science, particularly as applied in sexual assault investigation. He is also engaged in research on the comparative population genetics of staphylococci. He served on the two NAS Committees on DNA Technology in Forensic Science (1988–1992 & 1994–1996) and on the NAS Committee on Assessing the Research Program of the National Institute of Justice (2006–2010). He has served on the editorial boards of several forensic science journals. His professional memberships include the California Association of Criminalists, the American Academy of Forensic Sciences (Paul L. Kirk Award, 1987), and the International Society for Forensic Genetics (President, 18th International Congress, 1999). His graduate degree is from UC Berkeley.

Clifford Spiegelman, Ph.D. (statistics and applied mathematics), is Distinguished Professor of Statistics, Texas A&M University, College Station, TX, USA. His major research interests include applications of statistics to chemistry, proteomics, the environment, transportation, and the forensic sciences. He was a member of the National Academy of Sciences panel that evaluated the validity of comparative bullet lead analysis and published its findings as, Forensic Analysis: Weighing Bullet Lead Evidence (2004). He is the head organizer of the National Science Foundation's Statistics and Applied Mathematics Institute's 2015–2016 program on Forensic Science. His doctorate is from Northwestern University.

Hal Stern, Ph.D. (statistics), is the Ted and Janice Smith Family Foundation Dean and Professor in the Department of Statistics, University of California, Irvine, USA. His research interests include Bayesian methods, model diagnostics, forensic statistics, and statistical applications in biology/health, social sciences, and sports. He has authored more than 90 refereed publications and is a co-author of the highly regarded graduatelevel statistics text, Bayesian Data Analysis. He is a Fellow of the American Statistical Association and the Institute of Mathematical Statistics. He serves on the Physics/Pattern Interpretation Scientific Area Committee of the Organization of Scientific Area Committees (OSAC), a federal-standards setting body for forensic science. His doctorate is from Stanford University.

William C. Thompson, J.D., Ph.D. (psychology), is on the faculties of the Department of Criminology, Law & Society, the Department of Psychology and Social Behavior, and the School of Law (affiliated), at the University of California, Irvine, USA. He has published extensively on the use and misuse of scientific and statistical evidence in the courtroom and on jurors’ reactions to such evidence. His research has been funded by the National Science Foundation and the National Institute of Justice. He is a member of the Human Factors Subcommittee of the National Commission on Forensic Science and is Chair of the Human Factors Committee of the Organization of Scientific Advisory Committees (OSAC), a federal standards-setting organization for forensic science that is jointly sponsored by the U.S. Department of Justice and the National Institute of Standards and Technology (NIST).

James L. Wayman, Ph.D., (engineering), is in the Office of Research, San José State University, CA, USA. He has worked continuously in the field of automated human recognition since 1984. From 1997–2000, he was Director of the U.S. National Biometric Test Center. He has served on three National Research Council committees and is currently the Vice-Chair of the Forensic Speaker Recognition Subcommittee within the DOJ/NIST Organization of Scientific Area Committees. He is a Fellow of the IEEE and the IET and has 34 peer-reviewed publications. His Ph.D. is from the University of California, Santa Barbara.

Sandy Zabell, Ph.D. (mathematics), A.M. (biochemistry and molecular biology), is Professor of Mathematics and Statistics at Northwestern University, Evanston, IL, USA. He is a Fellow of the American Statistical Association and the Institute of Mathematical Statistics. He is currently a member of the Biological Data Interpretation and Reporting Subcommittee of the Organization of Scientific Area Committees of the National Institute of Standards and Technology, and is a member of the American Statistical Association's Ad Hoc Committee on Forensic Science. His graduate degrees are from Harvard University.

Ross E. Zumwalt, M.D., is a forensic pathologist at the New Mexico Office of the Medical Investigator and Professor of Pathology at the University of New Mexico School of Medicine, Albuquerque, NM, USA. His M.D. degree is from the University of Illinois College of Medicine; pathology residency at the Southwestern Medical School, Dallas; forensic fellowship training at the Dallas County Medical Examiner's Office; military service as director of laboratories at the Navy Regional Medical Center in Camp Lejeune, North Carolina: Deputy Coroner, Cleveland, Ohio (2 years); Deputy Coroner, Cincinnati, Ohio (6 years); Medical Examiner, State of New Mexico (1987–present); Chief Medical Examiner (1990–2014); certified in anatomic and forensic pathology by the American Board of Pathology; Trustee of the American Board of Pathology (1993–2004); President, American Board of Pathology (2000); President, National Association of Medical Examiners (1995–1996); Member, Committee on Identifying the Needs of the Forensic Science Community, National Academy of Sciences (2006–2009).

Footnotes

Committee on Identifying the Needs of the Forensic Science Community National Research Council, Strengthening Forensic Science in the United States: A Path Forward (2009) (hereinafter, NAS Report). The original, and parent, organization, created by Congress in 1863, during the administration of Abraham Lincoln, is the National Academy of Sciences. One of its major subunits is the National Research Council, through which ‘the NAS provides objective, science-based advice on critical issues affecting the nation’. http://www.nasonline.org (accessed August 28, 2016).

NAS Report, at 53.

Id. at 107, 108.

⁴

Brandon L. Garrett, Convicting the Innocent: Where Criminal Prosecutions Go Wrong 84, 117 (2010); Michael J. Saks & Jonathan J. Koehler, The Coming Paradigm Shift in Forensic Identification Science, 309 Science 892 (2005).

⁵

National Academy of Sciences, On the Theory and Practice of Voice Identification (1979).

⁶

National Research Council, Committee on Scientific Assessment of Bullet Lead Elemental Composition Comparison, Forensic Analysis: Weighing Bullet Lead Evidence (2004).

⁷

John Lentini, Fires, Arsons, and Explosions, in Modern Scientific Evidence: The Law and Science of Expert Testimony (Faigman et al. eds, 2010).

⁸

In high-profile cases, courts have even upheld convictions after DNA testing excluded the defendant, citing to the fact that a bitemark identification was made. See eg Brewer v. State, 819 So. 2d 1169, 1172, 1173 (Miss. 2002) (‘Dr. Michael West, the State's expert forensic odontologist, testified that it was his opinion that the bite marks on the victim were inflicted by Brewer’.). That expert did have testimony barred at a retrial in another case. See Steve Cannizaro, Buras Man May Beat Murder Rap Second Time, N.O. Times-Picayune, Dec. 21, 1996 at B1. It was rare for judges to even cite to Daubert (infra note 84) in rulings discussing any challenges to bitemark evidence. D. Michael Risinger, Navigating Expert Reliability: Are Criminal Standards of Certainty Being Left on the Dock? 64 Albany L. Rev. 99, 135–36 (2000).

⁹

Garrett, supra note 4, at 102, 105 (discussing DNA exonerations in cases of seven individuals, and how in five of those cases invalid testimony was presented in court claiming certainty that the defendant had left the marks in question).

¹⁰

The nearest any state has come to banning dental identification testimony is the Texas Commission on Forensic Science. ‘[C]oncluding that the validity of the technique has not been scientifically established’, the Commission has called for a moratorium on its use in court. Erik Eckholm, Texas Panel Calls for an End to Criminal IDs via Bite Mark, N. Y. Times, Feb. 12, 2016, http://nyti.ms/1o879OQ (accessed August 28, 2016). See, Texas Forensic Science Commission, Forensic Bitemark Comparison Complaint Filed by National Innocence Project on Behalf of Steven Mark Chaney - Final Report, Apr. 12, 2016.

¹¹

In re Richards, 63 Cal.4th 291 (2016).

¹²

C. Michael Bowers, Identification from Bitemarks, in Modern Scientific Evidence: The Law and Science of Expert Testimony (Faigman et al. eds, 2010).

¹³

Id., Sec. 37:1, note 2.

¹⁴

54 Cal. App. 3d 100, 126 Cal. Rptr. 350, 77 A.L.R.3d 1108 (2d Dist. 1975). An earlier case confronted the admissibility of expert testimony on a bite mark identification. Doyle v. State, 159 Tex. Crim. 310, 263 S.W.2d 779 (1954). This is not considered the seminal case for admission of bite mark evidence because of its peculiarities and lack of a following by courts or forensic dentists. Doyle was charged with burglary. At the site of the burglary was found a piece of partially eaten cheese. After arresting Doyle, the sheriff asked him to bite a piece of cheese, which the suspect voluntarily did. A firearms examiner compared plaster casts of the two pieces of cheese to try to determine if the questioned and the known tooth marks had been made by the same person, and agreed that they had. The Texas Court of Criminal Appeals upheld the admission of this bite mark opinion testimony. The defense in Doyle did not contest admissibility by raising any issue of scientific validity, but instead raised only legal procedural challenges. Thus, the Doyle court had no occasion to address the scientific status of bite mark identification. Nevertheless, another Texas court relied on Doyle 20 years later as the basis for rejecting an appellant's contention that bitemark test results were of unproven validity. Patterson v. State, 509 S.W.2d 857 (Tex. Crim. App. 1974).

¹⁵

Gerry L. Vale et al., Unusual Three-Dimensional Bite Mark Evidence in a Homicide Case, 21 J. Forensic Sci. 642 (1976).

¹⁶

68 Cal. 2d 319, 66 Cal. Rptr. 497, 438 P.2d 33, 36 A.L.R.3d 1176 (1968).

¹⁷

That third issue is a common error that has since come to be known as ‘the prosecutor's fallacy’. William C. Thompson and E.L. Schumann, Interpretation of Statistical Evidence in Criminal Trials: The Prosecutor's Fallacy and the Defense Attorney's Fallacy, 11 L. & Hum. Behav. 167 (1987).

¹⁸

Marx, supra note 14 at 111.

¹⁹

Id. at 113.

²⁰

Query whether keeping one's speculative (data-free) assumptions and logic quiet, rather than exposing them to the fact finder, exempts the resulting expert conclusions from the Collins limitation. Or is it, perhaps, an even more troubling violation of the principle?

²¹

People v. Milone, 43 Ill. App. 3d 385, 2 Ill. Dec. 63, 356 N.E.2d 1350 (2d Dist. 1976).

²²

Even after being paroled after serving nearly 20 years in prison for murder, Milone continued to insist upon his innocence and continued to try to clear his name. He went to federal court to challenge the original admission decision—arguing that bitemark expert evidence failed under both the Frye and Daubert standards. Furthermore, he offered evidence of another murder victim found in the same area where the victim he was accused of killing had been found. An apparent bite mark from the second murder victim was linked to a suspect, Macek. The bite marks on the two victims in the two cases were judged by at least one forensic odontologist to be indistinguishable from each other. Lowell Levine, Forensic Dentistry: Our Most Controversial Case, in Legal Medicine Annual (Cyril Wecht ed., 1978). Macek signed (but later withdrew) a confession to having killed the victim for whose murder Milone had been convicted. Discussed in State v. Sager, 600 S.W.2d 541 (Mo. Ct. App. W.D. 1980). The Court of Appeals for the Seventh Circuit expressed sympathy with Milone's request, especially in light of the evidence presented of Macek, his victim, and his dentition, but declined to rule on the case for lack of a constitutional basis for granting relief as well as because principles of federalism precluded a federal court from reexamining issues of fact reserved to the state court. Milone v. Camp, 22 F.3d 693 (7th Cir. 1994).

²³

People v. Slone, 76 Cal. App. 3d 611, 143 Cal. Rptr. 61 (2d Dist. 1978).

²⁴

2004 WL 502617 (D. Mass. 2004), aff'd in part, vacated in part, 405 F.3d 66 (1st Cir. 2005).

²⁵

380 F. Supp. 2d 852 (E.D. Mich. 2005), aff'd in part, rev'd in part on other grounds, 485 F.3d 364 (6th Cir. 2007).

²⁶

Id. at 871.

²⁷

Id. at 876.

²⁸

Id. at 857.

²⁹

Andre Moenssens et al., Scientific Evidence in Civil and Criminal Cases 985 (4th ed. 1995).

³⁰

Iain Pretty & David Sweet, The Scientific Basis for Human Bite Mark Analyses—A Critical Review, 41 Sci. & J. 85, 85 (2001). See also Mary A. Bush & Peter J. Bush, Current Context of Bitemark Analysis and Research, in Bitemark Evidence: A Color Atlas and Text § 6-303 (Robert B.J. Dorion ed., 2010) (2d ed. 2010); Ademir Franco et al., The Uniqueness of the Human Dentition as Forensic Evidence: A Systematic Review on the Technological Methodology, 129 Int'l J. Legal Med. 1277 (Nov. 15, 2015); Iain A. Pretty & David J. Sweet, Digital Bitemark Overlays—An Analysis of Effectiveness, 46 J. Forensic Sci.1385 (2001); NAS Report, at 176; Paul Giannelli, Edward J. Imwinkelried & Joseph L. Peterson, Reference Guide on Forensic Identification Expertise, Federal Judicial Center, Reference Manual on Scientific Evidence (3d ed. 2011) (hereinafter, FJC Reference Manual); C. Michael Bowers, Identification from Bitemarks, inModern Scientific Evidence: The Law and Science of Expert Testimony (David L. Faigman et al. eds, 2014) (hereinafter, Modern Scientific Evidence Chapter).

³¹

The findings of studies testing bite mark examiners’ ability to correctly identify the source of bite marks are reviewed, infra. The text's allusion to forensic techniques ‘still being practiced’ refers to several forms of forensic science (voiceprint identification, comparative bullet lead analysis, and a large number of arson ‘indicators’) that have ceased to be offered to courts following reviews by scientific bodies finding them to lack validity, though prior to those reviews they had frequently been admitted into evidence by courts.

³²

NAS Report, at 108.

³³

NAS Report.

³⁴

Harry T. Edwards, Solving the Problems that Plague the Forensic Science Community, 50 Jurimetrics J. 5 (2009).

³⁵

NAS Report, at 53.

³⁶

Id. at 176.

³⁷

Id. at 175. Though no scientific basis exists for identifying any particular person as the one and only possible source of a bite mark, such unwarranted assertions have been common in the testimony of forensic dentists. Illustrative of many other case are the following. In the capital rape-murder trial of Ray Krone in Arizona, two forensic dentists testified: ‘The teeth of Ray Krone did cause the injuries on the body of [the victim] to a reasonable degree of medical certainty. This represents the highest order of confidence that no other person caused the bite mark injuries’. ‘I'm certain [of the identification]’. (Figure 1 shows one of the evidence photographs from that case, comparing a mold of Krone's dentition to a bitemark on the murder victim. Ten years after being sentenced to death, Krone was exonerated by DNA.) At the Wisconsin trial of Robert Lee Stinson, a board-certified, ABFO diplomate concluded that the bite marks ‘had to have been made by teeth identical’ to Stinson's, and that there was ‘no margin for error’ in his conclusion. (After 23 years in prison, Stinson was exonerated by DNA.) At a preliminary hearing in Michigan, the forensic dentist testified that Anthony Otero was ‘the only person in the world’ who could have caused the bitemarks on the victim's body. (A month later, DNA testing excluded Otero as the perpetrator.)

³⁸

NAS Report, at 173.

³⁹

Id. at 174.

⁴⁰

Id.

⁴¹

See D. Michael Risinger et al., The Daubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion, 90 Cal. L. Rev. 1, 56 (2002); Itiel Dror et al., Contextual Information Renders Experts Vulnerable to Make Erroneous Identifications, 156 Forensic Sci. Int'l 74 (2006). The NAS Report called for further research regarding this problem.

⁴²

NAS Report, at 175.

⁴³

Id. at 175, 176.

⁴⁴

Allan Jamieson, The Philosophy of Forensic Scientific Identification, 59 Hastings L.J.1031 (2008).

⁴⁵

In regard to DNA, what we refer to as ‘markings’ or ‘marks’ would be equivalent to the visualizations of the DNA—at one time in the form of autorads, now as electropherograms.

⁴⁶

Research, described infra, suggests a high degree of interexaminer inconsistency among bitemark examiners.

⁴⁷

To say that every object of forensic interest is unique (that they can always be distinguished from each other, or that one can never be mistaken for another), are statements of speculation, not of empirical science. As a prominent population geneticist explained, ‘It is impossible to prove any human characteristic to be distinct in each individual without checking every individual, which has not been done’. David J. Balding, Weight-of-Evidence for Forensic DNA Profiles 54 (2005).

⁴⁸

At the same time, when one knows enough about the distribution of object attributes in the population, and the relevant probabilities in the case at hand are known (or believed on good grounds) to be sufficiently small, it is not irrational for a decision maker to conclude that the known and the questioned probably do share a common source.

⁴⁹

American Board of Forensic Odontology, Diplomates Reference Manual (January 2013), hereinafter referred to as the ABFO.

⁵⁰

Modern Scientific Evidence Chapter.

⁵¹

Dawn McQuiston & Michael J. Saks, Communicating Opinion Evidence in the Forensic Identification Sciences: Accuracy and Impact, 59 Hastings L.J. 1159 (2008).

⁵²

ABFO Diplomates Reference Manual (Jan. 2013).

⁵³

NAS Report , at 175.

⁵⁴

Id. (at numerous points in the report).

⁵⁵

ABFO Diplomates Reference Manual (2013), at 119.

⁵⁶

Id.

⁵⁷

Id. (emphasis in original). See also Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful Convictions, 95 Va. L. Rev. 1, 68 (2009) (pointing out that, despite forswearing insupportable extreme opinions, the ABFO guidelines allow ‘members to give conclusions expressing near certainty. Examples of the conclusions they may draw include that a bite mark matches a criminal defendant to a “reasonable medical certainty,” “high degree of certainty,” and “visual certainty with no reasonable possibility that someone else did it”.’).

⁵⁸

National Commission on Forensic Science, Testimony Using the Term ‘Reasonable Scientific Certainty’, U.S. Dep't of Justice (Apr. 2013). The proposed admonition apparently is aimed at witnesses and not courts because: ‘The Commission recognizes the right of each court to determine admissibility standards, but expresses this view as part of its mandate to ‘develop proposed guidance concerning the intersection of forensic science and the courtroom’.”

⁵⁹

Spencer S. Hsu, FBI Admits Flaws in Hair Analysis Over Decades, Wash. Post, Apr. 18, 2015, http://wapo.st/1OrujpH?tid=ss_mail (accessed August 28, 2016).

⁶⁰

FJC Reference Manual, at 104, 105.

⁶¹

Id. at 106.

⁶²

Under most circumstances, this distortion should lead to more false-negative errors than to false positives. On the other hand, if the bite mark has not been accurately recorded in the flesh, and will not match the actual biter, it sometimes can match, or be made to match (through manipulations used to ‘correct’ distortions), the dentition of other persons. R.G. Miller et al., Uniqueness of the Dentition as Impressed in Human Skin: A Cadaver Model, 54 J. Forensic Sci.909 (2009).

⁶³

NAS Report, at 174.

⁶⁴

Modern Scientific Evidence Chapter; see also NAS Report, at 174, 175; ABFO Diplomates Reference Manual (2013).

⁶⁵

For example, David Sweet & C. Michael Bowers, Accuracy of Bitemark Overlays: A Comparison of Five Common Methods to Produce Exemplars from a Suspect's Dentition, 43 J. Forensic Sci.362 (1998) (finding differences in accuracy as a function of method and recommending that forensic dentists cease using hand drawings of a suspect's teeth and increased use of digital images of dental characteristics).

⁶⁶

Eric S. Lander, Fix the Flaws in Forensic Science, N.Y. Times, Apr. 21, 2015 (arguing ‘[n]o expert should be permitted to testify without showing three things: a public database of patterns from many representative samples; precise and objective criteria for declaring matches; and peer-reviewed published studies that validate the methods’).

⁶⁷

NAS Report, at 174.

⁶⁸

Id.

⁶⁹

L. Thomas Johnson et al., Quantification of the Individual Characteristics of the Human Dentition, 59 J. Forensic Identification 609 (2009) (reporting one original study, observing that, ‘Very few studies have been published on the quantification of dental characteristics’, and noting that, ‘Expansion of the sample size through collaboration with other academic researchers will be necessary to be able to quantify the occurrence of these characteristics in the general population’.).

⁷⁰

Mary A. Bush et al., Statistical Evidence for the Similarity of the Human Dentition, 56 J. Forensic Sci. 118 (2011); H. David Sheets et al., Dental Shape Match Rates in Selected and Orthodontically Treated Populations in New York State: A Two Dimensional Study, 56 J. Forensic Sci. 621 (2011); Mary A. Bush et al., Similarity and Match Rates of the Human Dentition In Three Dimensions: Relevance to Bitemark Analysis, 125 Int'l J. Leg. Med. 779 (2011); H. David Sheets et al., Patterns of Variation and Match Rates of the Anterior Biting Dentition: Characteristics of a Database of 3D Scanned Dentitions, 58 J. Forensic Sci. 60 (2013). Measurement error, and thus the resolution of measurement of the dental arrangement, was quantified by repeated measurements of the same specimen, followed by analysis of the scatter of the measurement points. Resolution was determined to be 120 μm, or slightly more than one-tenth of a millimeter.

⁷¹

Sheets et al., Dental Shape Match Rates, supra note 70.

⁷²

Sheets et al., Patterns of Variation, supra note 70.

⁷³

See Gerald L. Vale et al., Unusual Three-Dimensional Bite Mark Evidence in a Homicide Case, 21 J. Forensic Sci. 642 (1976).

⁷⁴

The high error rates for bite mark identification, described infra, likely are in part caused by a tendency toward underguesstimation by forensic dentists of the probability that multiple members of a population will match a questioned bite mark.

⁷⁵

FJC Reference Manual, at 104.

⁷⁶

Simon A. Cole, Forensics Without Uniqueness, Conclusions Without Individualization: The New Epistemology of Forensic Identification, 8 L. Probability & Risk 233 (2009).

⁷⁷

NAS Report, at 174.

⁷⁸

Id. at 176.

⁷⁹

Ademir Franco et al., The Uniqueness of the Human Dentition as Forensic Evidence: A Systematic Review, 126 Int'l J. Legal Med. 1277 (2015).

⁸⁰

See supra note 60.

⁸¹

The most recent editions of the ABFO Diplomates Reference Manual state that the identification of a single biter from an open population of possible biters is no longer sanctioned.

⁸²

Even here, the rhetoric has again gotten ahead of any empirical research on the issues involved. Moreover, if investigators are mistaken about access being limited to all but the identified suspects, then we are back to an open population, only we don't know it. Furthermore, even the ‘closed population’ approach does not preclude errors of erroneously identifying an innocent suspect as the perpetrator. See the Gordon Hay case in Scotland. Case review presented at the 2000 meeting of the Forensic Science Society by Dr. Allan Jamieson.

⁸³

This approach to ‘accuracy’ comes from the field of signal detection theory. Propounded in the 1960s in such works as David M. Green and John A. Swets, Signal Detection Theory and Psychophysics 1 (1966).

⁸⁴

See Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 590 n.9 (1993) (discussing the distinction and stating, ‘In a case involving scientific evidence, evidentiary reliability will be based upon scientific validity’.) (emphasis in original).

⁸⁵

This is not a fanciful illustration. In the 1984 Forensic Sciences Foundation handwriting proficiency test of handwriting experts, all of the examiners taking the test independently reached the same conclusion that a particular writer was not the author of a particular questioned document (100 per cent reliability), but they were all incorrect (0 per cent validity). Summarized in D. Michael Risinger, Handwriting Identification, in Modern Scientific Evidence: The Law and Science of Expert Testimony (David L. Faigman et al. eds, 2013).

⁸⁶

These results were presented at the annual meeting of the 2015 American Academy of Forensic Sciences, held in Orlando, Florida, in February. ABFO officials have indicated that they do not wish the results published until further research has been conducted. However, the researchers supplied the raw data to a number of people, and we draw from their descriptions of it. The one published description is found in Radley Balko's, A Bite Mark Matching Advocacy Group Just Conducted a Study that Discredits Bite Mark Evidence, Wash. Post, Apr. 8, 2015, http://wpo.st/Rh5v1 (accessed August 28, 2016).

⁸⁷

David K. Whittaker, Some Laboratory Studies on the Accuracy of Bite Mark Comparison, 25 Int'l Dent. J. 166 (1975).

⁸⁸

Our description of the study and its findings is taken from the Modern Scientific Evidence Chapter on bitemark identification.

⁸⁹

Out of a possible maximum error rate of 27 per cent, examiners had a median overall error rate of 12.5 per cent, for an error rate that in effect was 46 per cent. Forensic dentist Michael Bowers, in Modern Scientific Evidence Chapter, explains why caution is needed in counting errors in such tests: Once one set of dentition is linked (correctly or incorrectly) to a bite mark, the others are not linked, and therefore are scored as ‘correct.’ In other words, given the test design, an examiner could never make more than two mistakes, and all remaining dentitions are scored as ‘correct’. If instead of providing a set of seven dentitions from which to choose, there had been 100, then the overall accuracy rate, using this seemingly straightforward method of counting, could never be lower than 98 per cent correct—one false positive inculpation of an innocent suspect, one overlooked guilty suspect, and 98 remaining dentitions that get scored as ‘correct’. And, thus, the poorest possible performance would be ‘2 per cent error’.

⁹⁰

Iain A. Pretty & David J. Sweet, Digital Bitemar k Overlays—An Analysis of Effectiveness, 46 J. Forensic Sci. 1385 (2001) (cautioning that the ‘[p]oor performance’ is a cause of concern because of its ‘very serious implications for the accused, the discipline, and society’, at 1390).

⁹¹

Sherie A. Blackwell et al., 3-D Imaging and Quantitative Comparison of Human Dentitions and Simulated Bite Marks, 121 Int'l J. Legal Med. 9 (2007).

⁹²

Mary A. Bush et al., Biomechanical Factors in Human Dermal Bitemarks in a Cadaver Model, 54 J. Forensic Sci. 167 (2009); Raymond G. Miller et al., Uniqueness of the Dentition as Impressed in Human Skin: A Cadaver Model, 54 J. Forensic Sci. 909 (2009); Mary A. Bush et al., The Response of Skin to Applied Stress: Investigation of Bitemark Distortion in a Cadaver Model, 55 J. Forensic Sci. 71 (2010); Mary A. Bush et al., Inquiry into the Scientific Basis For Bitemark Profiling and Arbitrary Distortion Compensation, 55 J. Forensic Sci. 976 (2010); H. David Sheets & Mary A. Bush, Mathematical Matching of a Dentition to Bitemarks: Use and Evaluation of Affine Methods, 207 Forensic Sci. Int'l 111 (2011); Mary A. Bush et al., A Study of Multiple Bitemarks Inflicted in Human Skin by a Single Dentition Using Geometric Morphometric Analysis, 211 Forensic Sci. Int'l 1 (2011); Hannah Holtkoetter et al., Transfer of Dental Patterns to Human Skin, 228 Forensic Sci. Int'l 61 (2013). These were the first studies in the bite mark field to investigate and summarize the biomechanical and structural properties of skin, including the J-shaped curve that describes the stress–strain relationship.

⁹³

To have physical properties that are different in different directions.

⁹⁴

The same conclusion was expressed recently by two prominent bitemark practitioners testifying about their casework: Frank Wright, testifying in State v. Prade, No. CR 1998-02-0463, 2013 WL 658266 (Ohio Com. Pl. Jan. 29, 2013), rev'd 2014-Ohio-1035, 9 N.E.3d 1072 (‘No two bite marks that I've ever seen from the same biter on the same victim look the same’.) David Senn, testifying in New York v. Dean, 04555 CR2007 (N.Y. Sup.Ct., June 12, 2012) (‘They are surprised… when the same teeth make bitemarks and they all look different, well we've known that forever’.). (Transcripts on file with author.)

⁹⁵

The scientific perspective is that fields’ claims are considered valid only to the extent that they have been empirically tested, using soundly designed research, yielding results that support the claims. That is also the perspective advanced by Daubert, supra note 84, as well by Frye v. United States, 293 F. 1013 (App. D.C. 1923) (though less explicitly than in Daubert).

⁹⁶

NAS Report, at 107, 108.

⁹⁷

See Pretty & Sweet, Critical Review, supra note 30, at 85.

⁹⁸

NAS Report, at 53.

⁹⁹

Federal Bureau of Investigation, FBI Clarifies Reporting on Microscopic Hair Comparisons Conducted by the Laboratory (July 13, 2012); see eg Texas Forensic Science Commission, Texas Hair Microscopy Case Review, http://www.fsc.texas.gov/texas-hair-microscopy-case-review (accessed August 28, 2016).

¹⁰⁰

Associated Press, Texas Reviewing Thousands of DNA Cases that Used Outdated Method for Calculating Odds, Dallas News, Jan. 31, 2016, http://www.dallasnews.com/news/crime/headlines/20160131-texas-reviewing-thousands-of-dna-cases-that-used-outdated-method-for-calculating-odds.ece (accessed August 28, 2016).

PERMALINK

Forensic bitemark identification: weak foundations, exaggerated claims

Michael J Saks

Thomas Albright

Thomas L Bohan

Barbara E Bierer

C Michael Bowers

Mary A Bush

Peter J Bush

Arturo Casadevall

Simon A Cole

M Bonner Denton

Shari Seidman Diamond

Rachel Dioso-Villa

Jules Epstein

David Faigman

Lisa Faigman

Stephen E Fienberg

Brandon L Garrett

Paul C Giannelli

Henry T Greely

Edward Imwinkelried

Allan Jamieson

Karen Kafadar

Jerome P Kassirer

Jonathan ‘Jay’ Koehler

David Korn

Jennifer Mnookin

Alan B Morrison

Erin Murphy

Nizam Peerwani

Joseph L Peterson

D Michael Risinger

George F Sensabaugh

Clifford Spiegelman

Hal Stern

William C Thompson

James L Wayman

Sandy Zabell

Ross E Zumwalt

Abstract

INTRODUCTION

I. LEGAL ORIGINS OF BITEMARK OPINION ADMISSIBILITY

II. GROWING DOUBTS

III. THE LOGIC OF FORENSIC IDENTIFICATION − −GENERALLY

A. Problems with declaring a ‘match’

B. Evaluation of an inclusion

IV. BITEMARK IDENTIFICATION IN LIGHT OF THE LOGIC OF FORENSIC IDENTIFICATION

A. The source of the bite mark

Figure 2.

Figure 1.

B. The substrate onto which a bite pattern is transferred

Figure 3.

Figure 4.

C. Methods of comparison

D. Lack of data on population frequencies

E. Uniqueness

V. HOW ACCURATE ARE BITEMARK IDENTIFICATIONS?

A. Measuring error—generally

B. Recent research on reliability

C. Studies of forensic dentists’ accuracy in simulated bitemark lineups

D. Studies of bite marks in a cadaver model

CONCLUSION

APPENDIX

Footnotes

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases