Discussion of “The Evaluation of Forensic DNA Evidence”^,

As our excerpt from the Executive Summary of this study states: In 1992, a broad-ranging report (1) released by the National Research Council attempted to explain the basics of the relevant science and technology, to offer suggestions for improving forensic DNA testing and its use in law enforcement, and to quiet the controversy that had followed the introduction of DNA profiling in court. Yet, the report did not eliminate all controversy. Indeed, in pro-pounding what the committee regarded as a moderate position—the ceiling principle and the interim ceiling principle—the report itself became the target of criticism from scientists and lawyers on both sides of the debate on DNA evidence in the courts. Moreover, some of the statements in the 1992 report have been misinterpreted or misapplied in the courts.

The current study began in 1994 as an update to the 1992 report on the issues of (i) the accuracy of laboratory determinations and (ii) the accuracy of calculations based on population and genetics theory and the available databases and the statistical assessments of similarities in DNA profiles.

In the end, again from our excerpt, “ … the committee agrees with many of the recommendations of the 1992 report but disagrees with others.”

The existence of two reports, close in time, which disagree on aspects of methodology illustrates what scientists have always known but what the law sometimes wishes to ignore: that scientists can differ in their expert judgment of the accuracy of the numbers produced from data by model-based formulae.

In this case the main focus of disagreement (Recommendations 4.1–4.4) is on the question of the extent to which models of population genetics can be applied in estimating the probability that the DNA of the suspect and DNA found on the victim match perfectly at each and every one of a preselected set of loci. This probability has to be computed under the assumption the match occurred “by chance alone.” That assumption is not enough to enable us to compute or rather estimate this probability. To finally arrive at a formula, further assumptions are made: treating the FBI or other data bases effectively as random samples from the relevant population and, more significantly, that Hardy Weinberg and linkage equilibriums are satisfied or are perturbed in a correctible way. Given that no laboratory error has been committed, there is, I believe, little disagreement between the committees or within the scientific community that the match probabilities referred to above are small, typically of order smaller than 1 in 1,000. But many scientists would not agree that the modeling assumptions made above can be verified to hold so precisely that the match probabilities can be ascertained to an order of 1 in one billion. The 1996 committee maintains that current models and data can be used down to that scale. The 1992 committee was more conservative but in a way that was challenged on scientific grounds by the 1996 committee.

The 1992 and 1996 reports essentially agree that laboratory error considerations be kept separate from match probability estimation and stress the importance of saving part of the sample for duplicate tests. In my view, when duplication is not possible, the chance of laboratory error, though difficult to ascertain exactly, appears to be of a larger order of magnitude than the probabilities discussed above. This issue is skirted around in both reports (Recommendations 3.1 and 3.2). The 1996 report includes discussions of important general questions of the interpretation of statistical evidence such as the prosecutor’s fallacy as well as issues such as searching databases and dealing with mixtures of DNA from several sources, a possibility graphically brought home by the O.J. Simpson case, which coincided with the latter half of the 1996 committee’s work (Recommendations 5.1–5.3).

In a final chapter on DNA evidence in the legal system, the 1996 committee makes the telling point that … the courts have demanded a more convincing showing of the exact degree of individualization yielded by DNA tests than any other commonly used forensic technique. Some courts have deemed it necessary for experts not only to demonstrate that DNA profiles usually vary from one person to another but also to produce uncontroversial, quantitative estimates of how rare the identifying characteristics are within particular groups and subgroups. Whether many other forms of identification evidence could survive comparable demands is doubtful.

This 1996 committee lays out its rationale for its recommendations very clearly and in considerable detail. We invite the readers of the Proceedings to make their own judgment on this analysis of a fascinating encounter between science and the law.