Machine learning enthusiasts should stick to the facts. Response to Morrison et al. (2022)

Dear Editor

In their letter [2], Professor Morrison and his co-authors comment on my recent article [1] in which I critically expose “machinism” as a novel form, among others, whereby current forensic science literature perpetuates unsuitable forensic source attribution claims (i.e., individualization). In the first paragraph of their commentary, the authors note that they “agree wholeheartedly” [2] with my general point that the persistence of source attribution claims in forensic science literature is problematic. In the second paragraph, the authors comment on my criticism of attempts to apply machine learning (ML) to forensic individualization. Here, too, Morrison et al. [2] agree with my point that ML not only bears potential for misapplication for the purpose of forensic source attribution, but that such misapplications actually do occur.

Besides our agreement in principle on all substantive points, Morrison et al. [2] raise concerns that neither reflect what I intended to convey nor what I actually wrote in my paper [1]. Specifically, Morrison et al. [2] call my account of ML a strawman-argument. This is incorrect because the type of forensic ML application that I discuss is not a misrepresentation, but real, and – as noted above – actually agreed by Morrison et al. [2]. The commentators further note that they “fear that readers will get the impression from Biedermann [1] that this [the use of the standard ML setting] is the only way (or at least the primary way) that machine learning is applied to forensic inference.” This fear is unjustified because my paper [1] makes no claims about the entirety of ML. On the contrary, I focus on a specific, well-defined and common ML template. It would require an intentional mind contortion to think that, what is a common computational architecture in general ML literature, is either the only or the primary way of using ML in forensic science. In the last sentence of their commentary, Morrison et al. [1] note that “there is in fact a growing body of literature on the calculation of well-calibrated likelihood ratios using machine-learning methods and relevant data, and on the validation under casework conditions of such machine-learning-based systems.” This note is a truism, but fails to make a point. The fact that ML procedures of some sort may be used as part of likelihood ratio (LR) inference procedures is immaterial to (and not targeted by) my critique of the misapplication of the standard ML scheme for the purpose of forensic source attribution decisions. What is more, and contrary to what I think Morrison et al. [2] suggest, probabilistic (i.e., LR-based) methods are actually mentioned multiple times in my paper [1], i.e., on pages 6, 7, 9 and 12. So, while I have a lot of sympathy with and welcome the challenge of my argument by machine learning enthusiasts and so-called “data scientists”, they should stay within the scope of the original argument in my paper [1], rather than inferring unstated motivations and conjecturing about what the reader might unjustifiably extrapolate from my paper. Learning relies on precise communication on the basis of what discussants say or do not say. This should bring to light where ML can go fundamentally wrong in forensic science.

Declaration of interest

The author declares that this letter to the editor was written in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.