Science in the court: pitfalls, challenges and solutions

Abstract

We are at a critical juncture for the forensic sciences. A number of high-profile reports and a growing body of literature question and critically reflect on core issues pertaining to the methodologies informing forensic science and their effective use within the criminal justice system. We argue for the need for an improved association and outline key areas that require attention from practitioners operating within the fields of both forensic science and law.

1. Introduction

Increasingly, academics and legal practitioners are adopting a more critical attitude towards the forensic sciences, the way they are introduced at trial and the role they play in convictions or acquittals once admitted. Moreover, practitioners from a range of forensic science specialties have begun a reflective appraisal, wherein the shortfalls and indeed validity of a number of heretofore accepted techniques are being questioned. De facto deference to the weight ascribed to forensic evidence in the courtroom or indeed to the opinions of experts is being consigned to the past. Shortfalls inherent in the current system include operational problems related to the efficiency of the justice system and the way it is administered [1], the admissibility of expert evidence [2], reliability tests [3] and structural problems including the influence of the evidence tendered by experts on the jurors [4], the adversarial nature of the system in common law jurisdictions [5], the bias of legal representatives [6] and flawed assumptions in forensic sciences [7–10].

The benefits yielded from the output of the forensic sciences [11]¹ and associated utility for prosecutions or acquittal of the perpetrators of crime has meant that scientific evidence has become a mainstay in the functioning of our criminal justice system. While increasingly the techniques of the forensic sciences are of relatively contemporary origin, the premise behind using physical or material evidence and their application in a legal forum has a much longer history. It would appear that the affiliation between human testimony and material evidence has long since been beset with an underlying tension. As Eyal Weizman [12] highlights, ‘the history of jurisprudence tells of a constant tension between human testimony and material evidence, and an ongoing shift of emphasis, at different periods, between them’ (p. 105). Moreover, the fruits of forensic evidence remain latent for the purposes of justice until an act of conversion takes place, or in other words an interlocutor is required to interpret and explain the significance of any piece of potential evidence.

Derived from the Latin forensis, the word's root refers to the ‘forum’, and the practice and skill of making an argument before a professional, political, or legal gathering. In classical Rome forensics was part of rhetoric, which of course concerns speech. However, forensics included not only human speech but that of objects. In forensic rhetoric, objects could address the forum. Because objects do not speak for themselves, there is a need for ‘translation’ or ‘interpretation’—forensic rhetoric requires a person (or a set of technologies) to mediate between the object and the forum: to present the object, interpret it and place it within a larger narrative. This was the role of the rhetoricians and today it is the role of the expert witness. [12, p. 105]

Thus, when unpacking the association between science and the law, our critical gaze turns not only to the material or forensic evidence, but also to the medium through which the evidence is presented to the forum—the expert witness—and how this dialogue is received and perceived by the audience in the court, comprising both legal practitioners and lay members of the public. Our concern with initiating a paradigm shift should, as a consequence, focus on the robustness and admissibility of the evidence that comes before the court, the suitability of the expert, and the integrity and effectiveness of this act of active translation.

2. A critical turn: the challenges facing forensic evidence and its role in our courts

A critical juncture in the confidence bestowed on forensic sciences within a criminal justice context can be identified in a growing body of literature and a number of high-profile reports that question core issues including how evidence is collected from crime scenes, how it is collated and interrogated in the laboratory setting, the manner in which evidence is introduced at trials, how it is ultimately interpreted and evaluated within a case context, and the role it plays in convictions or acquittals once it has been admitted (cf. [11,13–16]). Challenges and pitfalls that have been identified in relation to the introduction of forensic evidence in the courtroom range across both operational problems and structural shortcomings. Tension points in the relationship between science and the law are exemplified in high-profile cases, including the convictions in Cannings,² Dallagher,³ Clark⁴ or Harris.⁵ The work of rights-based movements including the Innocence Project are premised, in part, on an underlying mistrust of the reliability of expert evidence introduced at trial and the role that forensic evidence, inter alia, plays in wrongly convicting the innocent, or indeed conversely contributes to a failure to convict the guilty [17,18]. Ironically, ‘innocence fraud’ is now being discussed under the auspices of an Innocence Audit [19].

While the systematic difficulties that serve to undermine the relationship between forensic science and the law should not be reduced to a discourse on miscarriages of justice, it is perhaps this phenomenon that has become most emblematic of the breakdown in communication and perhaps therefore is the obvious mechanism by which a paradigm shift might be effected. The cost of a troubled relationship arguably becomes most apparent in the light of such failure to deliver justice (e.g. [7,10,20–24]. Key texts [25,26] point to procedural errors that may lead to a wrongful verdict: a failure to provide the jury with key information due to negligence [13], bodies in charge of ensuring the quality of forensic services that do not have effective quality control procedures in place to check adequately the evidence tendered before being included in trials [26], the use of non-unique identifiers for the exhibits or samples tested [25], the use of checking procedures relying on non-unique identifiers [25], ineffective communication between the bodies responsible for gathering and checking the evidence [25], and lack of proper implementation of handling procedures [25]. It has also been pointed out [13] that there is a need to guard against the so-called ‘escalation effects’—when one such procedural or human error occurs, this can lead to additional cumulative errors potentially culminating in a wrongful conviction. Although not uncomplementary to each other in an operational sphere, forensic science and the law represent epistemologies from different disciplinary traditions and different understandings of truth can inform the work of both scientific and legal practitioners. If the starting points are not the same, it cannot be assumed that the end-point for these traditions, when they meet in the court room, will be the same. And indeed such divergence can give rise to significant operational challenges. As the Daubert⁶ decision notes, ‘there are important differences between the quest for truth in the courtroom and the quest for truth in the laboratory. Scientific conclusions are subject to perpetual revision. Law, on the other hand, must resolve disputes finally and quickly’ (p. 2798). While the relationship between science and law is a complex one, with multifaceted factors contributing to the current state of play, we outline here three issues that, in our opinion, could be addressed if we are to seek an improved, and healthier, association. This is by no means an exhaustive list.

(a). Admissibility of expert evidence

In England and Wales, a somewhat laissez-faire approach to the admissibility of expert evidence has been reported, and it is rare for any forensic evidence to be deemed inadmissible, unless it is glaringly inappropriate [16]. This approach is justified by the parties concerned through the conviction that the reliability of any piece of expert evidence will be challenged effectively during the trial, either by cross-examination or by the opposing party's expertise. The deficit in this approach is wryly noted by Schauer [27], who remarks: ‘The scientist who seeks to determine whether drinking red wine reduces the likelihood of heart disease does not, for example, summon representatives of the wine industry and the Temperance League to each make their cases and thereafter decide which of the two advocates is more believable’ (p. 1193).

Looking to other common law systems, when compared to the Frye and Daubert standards that have been imposed in the USA, this method of approaching the admissibility of expert evidence seems fundamentally flawed and almost an abrogation of responsibility. The Frye standard, established as a result of the judgment in Frye v. United States,⁷ demands that, in order for the results of a scientific technique and the subsequent evidence to be admissible, that technique must have gained general acceptance in its particular field [28]. The Daubert standard, as instituted via the Daubert v. Merrell Dow Pharmaceuticals⁶ judgment, requires the trial judge to screen the scientific evidence and testimony to ensure that it is relevant and reliable [28]. Although highly preferable to the laissez-faire approach to scientific evidence in England and Wales, these two standards have their own shortcomings. With Frye, that a practice is accepted in a scientific community is not proof of its quality or veracity. The researcher devising a novel practice might initiate what Kuhn termed a ‘scientific revolution’, which necessitates working outside an entrenched scientific paradigm. The Law Commission agreed that this type of emergent scientific practice should not be excluded in principle as courtroom testimony but scientific safeguards must be appropriate to ensure that evidence does not stray beyond the reasonable limits of the approach. With Daubert, screening scientific evidence requires a level of scientific preparation and knowledge background that is not usually in a judge's repertoire. It was a recognition of the problems inherent with the admissibility and understanding of expert evidence in criminal proceedings and an acknowledgement of a need for reform that prompted the Law Commission's consultation and review process in England and Wales.

(b). Reliability factor

The reliability factor is a further challenge which must be considered. This stems from decisions in cases such as Reed⁸ or Atkins,⁹ where the court admitted evidence that would, by the Law Commission's test, be considered unreliable because of potential methodological issues despite them being made clear to the jury. For example, judgments such as Reed, R. v T¹⁰ and Atkins demonstrate the specific problems associated with a lack of ground truth databases for some of the branches of forensic science used in these cases. The reliability of forensic sciences that do not permit statistically quantified opinions to be expressed is a well-documented topic. In cases such as this, there is general disagreement between scientists and legal representatives and also between academics and practitioners. Academics often advocate for a statistical quantification of expert opinion, claiming that it is a factor that determines the reliability of any branch of real science [3,14,29–31]. For their counterparts, there is a view that such a standard is plainly unnecessary or even undesirable, given the challenging nature of understanding statistics for many legal practitioners and jury members.

Starting from unreliable expert opinion in cases such as Cannings, Clark or R. v T, the problem of determining reliability has gained prominence, either as a result of the expert expressing opinions outside their own field, straying beyond the boundaries of their own expertise or relaying it in a form of a quantified scale that has no sound scientific basis. The need for a standard to determine whether an expert opinion is robust, worthy of consideration or merely an attempt to serve the views of the party the expert is representing arises especially in connection with forensic evidence types that are traditionally highly reliant on comparative and therefore largely subjective analysis: e.g. fingerprint analysis, footwear mark analysis, handwriting, ballistics, fibres or hair. In certain cases, the unreliability of these opinions has prompted a demand for standards that are verifiable and grounded in available data, and this has proved problematic.

The introduction of a developed mathematical approach stemming from the application of Bayes theorem represented a disruptive and profound change to evaluating the meaning of scientific evidence as it was applied to the set of circumstances relating to an alleged criminal act. The framework that has emerged over many years now provides a means of addressing competing hypotheses (prosecution and defence) and emphasizes the necessity to demonstrate robust, logic, transparent and balanced thinking [29]. This can involve [3]

(1) the use of the likelihood-ratio framework for the evaluation of evidence; (2) a strong preference for objective measurements, the use of databases representative of the relevant population, and the use of statistical models; and (3) testing of the validity and reliability of the forensic-comparison system under conditions reflecting those of the case at trial. (p. 2)

Of course, it is also certainly recognized that with the possible exception of DNA, the development of both representative and ground truth databases has been financially challenging to say the least. The implementation of the likelihood-ratio approach for DNA evidence and more latterly other evidence types (particularly trace evidence) has facilitated the development of new thinking in case management in relation to an overarching case assessment and interpretation approach, where forensic science is used in a holistic manner rather than as a series of deployed siloed tests.

Certain trial appeals and pieces of literature^8,9,11 [8,9,31–36] have focused on the dubious classification of a number of branches of forensic science as ‘science’. While they can be organized and recognized as constituting a unified body of knowledge, and the expert can draw on this to formulate an opinion as the Law Commission suggests, this does not guarantee a reliable or indeed scientific opinion. The specialties particularly under challenge are the so-called ‘soft sciences’, including footwear mark analysis and fingerprint analysis with more severe criticisms forthcoming for gait comparison and facial mapping. What differentiates these subjects from more accepted forensic expertise including DNA analysis, drug analysis or toxicology is the lack of a clear methodology and underpinning scientific research and rigour. For some of these subjects, there is a very arbitrary standard for expert qualifications, or indeed standards may be non-existent. Fingerprint analysis has probably the most clearly formulated standards, but yet these could, and have been, open to critique from a methodological perspective: in the past to qualify as a fingerprint expert, the practitioner need only complete the UK National Fingerprint Learning Programme, although in more recent years and post the Scottish Fingerprint Inquiry, competency tests have been introduced by some police forces. Other soft sciences, such as facial mapping, gait analysis or footwear mark analysis have no specific subject standards and usually, in order to be considered reliable as an expert, one has only to prove experience in the field. The statement in court ‘I know this to be true because I have seen hundreds of these cases’ should never be accepted.

Compounding this challenge further is the lack of comparable data or ground truth databases available for population representativeness, which would permit comparison of forensic samples within appropriate reference boundaries. This problem has been especially glaring in cases^8,9,11,12,13 where the experts have used sliding scales to quantify the level of support the science offers for a particular hypothesis. This is problematic in the absence of more representative population data, especially since the softer sciences may rely on minute observations of features or traits. These can be salient if the experts can prove that the co-occurrence of two or more of these details is rare in the population (but how rare?), but they could also be deemed insignificant in the absence of suitable comparator data. The problem is explained very well by Edmond et al. [31]:

The absence of a database or some other credible method of assigning significance to purported similarities means the observer has no reasonable basis on which to draw conclusions about identity. This makes it necessary to invent a value (like ‘lends strong support’), which is simply a subjective summation (or impression) lacking methodological rigour. (p. 3)

On this point, there is a clear disagreement, with legal academics and some practitioners claiming that this type of evidence ought to be admitted as it still assists the jury, while other scientific experts challenge the position as offering no solid base upon which to build an expert opinion. Issues such as this have been at the heart of many failed appeals [31], with the exception of R. v T. The decision of judges to deem the conviction safe has also achieved support and recognition in the literature [14,27,35,37,38] for various reasons including, ‘[T]he reliability and validity standards for scientific evidence that courts use must be standards that come, ultimately, from the legal goals of legal institutions and not from the scientific goals of scientific institutions’ [27, p. 1214].

These points are perhaps best illustrated with DNA analysis, the so-called ‘gold standard’ of forensic sciences [11]. Here, the scientific case is compelling, with a solid scientific acceptance of both the science and the technological solutions used to extract and analyse DNA profiles. Where the unrest currently lies is with the use of low template DNA and the interpretation of mixed and complex DNA profiles, where there are contributions from more than one individual or they represent degraded or partial profiles. Here the scientific debate is far from resolved, with differing views espoused in the literature. Ultimately, this has resulted in the recent rejection of this form of DNA evidence in both USA [39] and Australian courts. This is perhaps the ultimate example of previous commentary coming home to roost ‘as with other areas of expert evidence, not all DNA results are equal, with some being rather more equal than others’ [40]. The consequences of this ruling are far reaching, given that the gold standard has now (at least in two jurisdictions) been fundamentally questioned and rejected.

(c). Rapport between the expert and the jury

A further challenge for science in the courts is the rapport between the jury and the expert. Forensic evidence needs to be ‘[s]cientifically rigorous, but accessible’ [41, p. 2]. The Turner rule allows the court to exclude evidence, even if it is of probative value, if it is likely that it will influence the jury unduly on a matter on which they ought to form their own independent judgment [37]. It is a necessary way of limiting the admissibility of expert evidence given the fact that it is presented to lay people, who might defer to it too readily. An important consideration when reviewing the role of forensic evidence in the courts relates to the nature of the scientific evidence and the degree to which it influences the jurors—and there are three levels we can consider.

Introduced through expert evidence are sciences which require an empirical analysis of the data; ‘a method that relies on scientific instrumentation to provide quantitative data that can mathematically represent results’ [42, p. 4]. Examples of this are drug analysis, toxicology or DNA analysis. These are distinct for their quantitative approach to evidence and the fact that the result can often be codified, either as a ratio or as a number. The perception of the lay public is that these sciences are largely infallible: there is little room for human error, and objective results can be furnished by a computer. This is a position which is increasingly undermined with recent challenges to the validity of DNA analysis in court [43] or, for example, the increasing challenges to the identification of new psychoactive substances.

The forensic sciences that rely fundamentally on comparative analysis: ‘a method that involves the comparison of an item of evidence of unknown source against an item of known source or origin’ [42, p. 4] cause different issues. Forms of comparative analysis include fingerprint evidence, ballistics, footwear mark analysis or handwriting, and these are often-used ‘sciences’ in the courtrooms sometimes furnishing exculpatory results. Evidence types such as facial mapping or gait comparison also involve comparative analysis but are less well accepted. The problem is, nonetheless, that when a dispute exists between experts, there is no objective way to reach agreement. In other words, based solely on the science, a lay person has no means to decide which of two experts, expressing opposing viewpoints, is correct. These arguments are usually settled in courtrooms by exogenous methods, including ranking the experience, qualifications or reputation of the two experts, and dismissing one of the two opinions on the basis of relative inexperience.^8,12,13,14

Finally, yet other forensic sciences make use of interpretative analysis, which aims to ‘reconstruct’ the accident or crime based on ‘clues’ available at the scene. This approach is highly subjective and depends on the experience and intuition of the expert relative to a given case. Such a form of analysis has been used in Kwaik,¹⁵ where the expert used a computer program which, on the basis of input, could allegedly re-create a car accident.

The three approaches presented above are likely to have a progressive decreasing influence on the jury, since the first strand tends to be considered more objective and the last one tends to be regarded as being more questionable and subjective. But this is not to say that the last two categories have no value and do not influence the jury. They do so, either by means of an insufficiently transparent, and probably inadequate, methodology,¹⁶ or by means of expert credibility—typically based on an expert's experience, qualifications, reputation and performance. The fact that expert credibility has a significant influence on the jury can be seen from a number of cases where scientific debates in the court were adjudicated on this basis.^14,15 This way of influencing the jury seems nothing short of an ad hominem fallacy, where a claim is accepted or rejected on the basis of an irrelevant factor about its author. The problematic way of testing the science by means of testing the expert's background has also been raised by the jury in Tang,¹⁷ who pertinently asked the judge in connection with the expert testifying: ‘Accepting [the expert's] qualifications, should we also accept her methodology?’ (para. 50). Ideally, this should not be the case, even acknowledging how important experience and qualifications are, they only really contribute to the interpretation of evidence within a case context. The question is whether there is anything else of value, apart from perceived credibility.

Maybe expertise that seeks to conceal its lack of methodological rigour behind an expert with an impeccable reputation should be deemed no more than observational evidence. Consider also the case of newly emergent, but perhaps highly dubious scientific underpinning, including barefoot morphology [44] or brain fingerprinting [34], which have extensive intuitive appeal, but do not meet the criteria for classification as core scientific evidence. The significant gap between scientific practice and these techniques should not be hidden from the jury behind the credibility and persuasive performance of any expert. Accounting for this, it is perhaps the case that the influence expert evidence and the credibility of the experts themselves can have on the juries is very much dependent on what sort of evidence is admitted and in which light it is portrayed by the judge. Preferably, practices that fall short of methodological rigour should not be admitted at all, on account of their unreliability. However, a judge might hesitate in excluding such well-established practices as fingerprint analysis from trial: ‘procedurally, there is always a temptation to let evidence in, fully understood or not, and for the judge to leave it until the end of the trial to determine what weight, if any, is to be given to it’ [45, p. 8]. Consequently, the judge should assume the function of guiding the jury and attenuating the disproportionate impact this sort of evidence may have on the jurors. The judge makes clear the limitations of the respective scientific practice, assessed against a scientific, not a legal, standard, at the same time managing the expectations that jurors may have from any piece of scientific evidence during trial by defusing the so-called ‘CSI effect’. In this way, the judges assume the role of a ‘gatekeeper’. To be competent in this role, the judge is required therefore to remain current with scientific advances but they are rarely qualified to do so and indeed the vast array of subject matter makes this a Herculean task. In the words of the Lord Chief Justice [41]:

Scientific developments, inconceivable 20, let alone 50 years ago, come with risks: the pace of change and complexity of techniques present challenges for all involved, not least those of us without scientific backgrounds. The court must be satisfied that there is sufficiently reliable scientific basis for the evidence to be admitted. (pp. 2–3)

3. Solutions

For the challenges highlighted above, there are practical solutions which can be proposed to address the current inadequacies. The recommendations made in the NAS report are a good reference point and it has been observed that they ‘are just as applicable to our system in the UK as they are to the US’ [46, p. 4]. More specifically, for England and Wales, the Lord Chief Justice [41] proposed several key measures for improvement relating to consolidating the ‘reliability of the underlying science’ (p. 6), proposed means by which the quality and integrity of expert witnesses can be ensured and considered means by which forensic expertise can be introduced at trial in a manner where it can be understood. For the latter concern, this is an important component for realizing a significant step change for our current system, because a strengthening of the underlying science and the expert witness selection, through accreditation for example, would render only limited gains without providing a way in which expert evidence can be understood by those who are called upon to determine a defendant's culpability at trial. One approach might be to introduce ‘standardized documents’ or ‘primers’ which would have the benefit of facilitating the presentation of basic science in an accessible format. These primers, restricted to branches of forensic science for which there existed a consensus amongst scientists, would in the opinion of the Lord Chief Justice ‘assist juries in understanding the concepts underpinning the issues in their case’ (p. 13). One might extrapolate that the absence of a primer for a particular discipline might be a stark indicator of the value of that evidence type for the legal teams and the triers of fact.

This, however, still exposes the need for forensic science to be honest with itself and reflect on the fundamental deficiencies within its ranks. The scientific community needs to address these to merit judicial confidence or else be brave enough and have sufficient professional conscience to abandon the evidence types that do not meet the required judicial reliability before they are forced to do so in open court. Furthermore, the onus is on the forensic science and judicial community to work in tandem to ensure that the introduction of new technologies, essential to keep pace with the developing world, are managed in such a way as to ensure judicial confidence from the outset. Finally, the challenge is also laid before the forensic science and law enforcement communities to allow the scientists to undertake their craft and use their skills in the correct way, working in partnership to inform the investigative process rather than being downgraded to act simply as laboratory technicians who churn out data and report their results in a contextualized vacuum.

One of the overarching themes which emerges from the current forensic science ecosystem is that of poor communication. It is critical that the legal participants and jury understand the underpinning scientific rationale of the evidence placed before them and can have confidence in its scientific rigour. Armed with this basic knowledge, they are informed to translate and contextualize such evidence to the case within which it is presented.

4. Conclusion

In his address on the theme of expert evidence and the future of forensic science in criminal trials, delivered in late 2014 for the annual Criminal Bar Association Kalisher lecture, the Lord Chief Justice of England and Wales [41] posed a rhetorical question, asking ‘[w]hy is forensic science a relevant and topical subject’ (p. 2). Likewise, we may ask, why is forensic science and its role in criminal trials a relevant subject for the focus we have undertaken in this paper? To quote The Lord Chief Justice, it is important for four core reasons:

First … we have always faced problems. Second, there are current problems. Third, we have had excellent proposals for reform from the Law Commission which have been largely implemented in a novel way. Fourth, embedding the reforms and addressing other problems is important for overall public confidence in criminal justice. It will show the public why justice matters at a time when there is a risk that justice will be overlooked as there are so many other calls on the public purse. (p. 2)

Unsafe forensic evidence and unclear boundaries to guide forensic expert testimony in the courts have the potential to undermine a system which can lead to grave consequences, including miscarriages of justice. Our system of criminal justice works under cherished principles including the right to fair trial, and the tools of forensic science must be sharpened to work for the protection of such rights, not designed to work against it.

There is no disagreement that forensic science is a mainstay in the effective delivery of a fair and equitable justice system and no disputing its importance in the resolution of legal dispute. Yet, the relationship between science and the law can at best be described as a firmly rooted confrontational symbiotic tolerance. One of the first documented clashes occurred when Galileo promulgated to a highly charged and hostile court that, in his opinion, ‘careful laboratory experiments could reveal universal truths’. He was not well received.

We must learn from the past, disrupt our thinking, strengthen the community and change our culture. This means working together towards a resolution of the scientific deficiencies within existing forensic evidence while providing a firm basis for new innovative technologies entering into the forensic science ecosystem. At the same time, we need to ensure that the law enforcement and investigative communities once again recognize and use forensic science to its full potential as a holistic problem-solving tool (for example, through the use of the case assessment and interpretation methodology [47–50]). That such a methodology is embedded within a framework which allows for an understanding of the contribution that a specific evidence type could meaningfully deliver in terms of sub-source, source, activity or offence-level propositions for a given set of case specific circumstances rather than restricting it to a siloed one-dimensional reactive process becomes obvious. A contextualized means of evaluative reporting of forensic science data pertinent to a particular case but held in the context of that case where alternative propositions can be attended to and challenged correctly, works to fulfil this problem-solving potential. Such an approach has been suggested by the Association of Forensic Science Providers [32] among others.

The demands currently confronting forensic science are recognized as a broader, global challenge, arising from structural and operational shortcomings common across a number of jurisdictions (for example, in the UK, USA [11], The Netherlands [51] and Australia), as well as from contemporary manifestations of criminality, including terrorist acts [52]. This calls for us to respond to transitional challenges, in addition to supporting a more far reaching paradigm shift for forensic science.

There is no doubt that forensic science is in crisis, and it currently faces its most uncertain future. However, our future is in our own hands and what we, as a criminal justice community, choose to do next will be our legacy.

Competing interests

We declare we have no competing interests.

Funding

Posts of Anuta Musat and Tasneem Sadiq were supported under the University of Essex Frontrunner Plus scheme and the Undergraduate Research Opportunity Programme.