Abstract
We agree wholeheartedly with Biedermann (2022) FSI Synergy article 100222 in its criticism of research publications that treat forensic inference in source attribution as an “identification” or “individualization” task. We disagree, however, with its criticism of the use of machine learning for forensic inference. The argument it makes is a strawman argument. There is a growing body of literature on the calculation of well-calibrated likelihood ratios using machine-learning methods and relevant data, and on the validation under casework conditions of such machine-learning-based systems.
Keywords: Forensic inference, Machine learning
Letter to Editor:
Biedermann [1] is critical of research publications that treat forensic inference in source attribution as an “identification” or “individualization” task. Biedermann [1] argues that such publications condone unscientific attitudes and practices, foster unrealistic expectations among consumers of forensic science, and undermine trust in peer-reviewed publications because so-called “original research papers” are not, in fact, well grounded. With respect to these points, we agree wholeheartedly with Biedermann [1].
With respect to criticism of machine learning, however, we feel that Biedermann [1] makes a strawman argument. It defines “standard” machine learning as outputting categorical decisions and then criticizes the use of “standard” machine learning for forensic inference because it outputs categorical decisions. There are indeed research publications that misapply machine learning to forensic-inference problems, including using algorithms that output categorical decisions, e.g. [2]. But we fear that many readers will get the impression from Biedermann [1] that this is the only way (or at least the primary way) that machine learning is applied to forensic inference. There is in fact a growing body of literature on the calculation of well-calibrated likelihood ratios using machine-learning methods and relevant data, and on the validation under casework conditions of such machine-learning-based systems. Recent examples include [[3], [4], [5], [6], [7], [8], [9], [10], [11]].
Disclaimer
All opinions expressed in the present paper are those of the authors, and, unless explicitly stated otherwise, should not be construed as representing the policies or positions of any organizations with which the authors are associated.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Author contributions
Morrison, Ramos, Ypma: Writing - Original Draft, Writing - Review & Editing. All other authors: Writing - Review & Editing.
Acknowledgements
The writing of this response was supported by Research England's Expanding Excellence in England Fund as part of funding for the Aston Institute for Forensic Linguistics 2019–2023.
References
- 1.Biedermann A. The strange persistence of (source) “identification” claims in forensic literature through descriptivism, diagnosticism and machinism. Forensic Sci. Int.: Synergy. 2022;4 doi: 10.1016/j.fsisyn.2022.100222. article 100222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Quijano-Sánchez L., Liberatore F., Camacho-Collados J., Camacho-Collados M. Applying automatic text-based detection of deceptive language to police reports: extracting behavioral patterns from a multi-step classification model to understand how we lie to the police. Knowl. Base Syst. 2018;149:155–168. doi: 10.1016/j.knosys.2018.03.010. [DOI] [Google Scholar]
- 3.Bosma W., Dalm S., van Eijk E., El Harchaoui R., Rijgersberg E., Tops H.T., Veenstra A., Ypma R.J.F. Establishing phone-pair co-usage by comparing mobility patterns. Sci. Justice. 2020;60:180–190. doi: 10.1016/j.scijus.2019.10.005. [DOI] [PubMed] [Google Scholar]
- 4.Matzen T., Kukurin C., van de Wetering J., Ariëns S., Bosma W., Knijnenberg A., Stamouli A., Ypma R.J.F. Objectifying evidence evaluation for gunshot residue comparisons using machine learning on criminal case data. Forensic Sci. Int. 2022;335 doi: 10.1016/j.forsciint.2022.111293. article 111293. [DOI] [PubMed] [Google Scholar]
- 5.Morrison G.S., Enzinger E., Hughes V., Jessen M., Meuwly D., Neumann C., Planting S., Thompson W.C., van der Vloed D., Ypma R.J.F., Zhang C., Anonymous A., Anonymous B. Consensus on validation of forensic voice comparison. Sci. Justice. 2021;61:229–309. doi: 10.1016/j.scijus.2021.02.002. [DOI] [PubMed] [Google Scholar]
- 6.Morrison G.S., Enzinger E., Ramos D., González-Rodríguez J., Lozano-Díez A. In: Handbook of Forensic Statistics. Banks D.L., Kafadar K., Kaye D.H., Tackett M., editors. CRC; Boca Raton, FL: 2020. Statistical models in forensic voice comparison; pp. 451–497. Ch. 20. [DOI] [Google Scholar]
- 7.Ramos D., Meuwly D., Haraksim R., Berger C.E.H. In: Handbook of Forensic Statistics. Banks D., Kafadar K., Kaye D.H., Tackett M., editors. CRC; Boca Raton, FL: 2020. Validation of forensic automatic likelihood ratio methods; pp. 143–163. Ch. 7. [DOI] [Google Scholar]
- 8.Ramos D., Maroñas J., Almirall J. Improving calibration of forensic glass comparisons by considering uncertainty in feature-based elemental data. Chemometr. Intell. Lab. Syst. 2021;217 doi: 10.1016/j.chemolab.2021.104399. article 104399. [DOI] [Google Scholar]
- 9.Vergeer P., Hendrikse J.N., Grutters M.M.P., Peschier L.J.C. A method for forensic gasoline comparison in fire debris samples: a numerical likelihood ratio system. Sci. Justice. 2020;60:438–450. doi: 10.1016/j.scijus.2020.06.002. [DOI] [PubMed] [Google Scholar]
- 10.Weber P., Enzinger E., Labrador B., Lozano-Díez A., Ramos D., González-Rodríguez J., Morrison G.S. Validation of the alpha version of the E3 Forensic Speech Science System (E3FS3) core software tools. Forensic Sci. Int.: Synergy. 2022;4 doi: 10.1016/j.fsisyn.2022.100223. article 100223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Ypma R.J.F., Maaskant-van Wijk P.A., Gill R., Sjerps M.J., van den Berge M. Calculating LRs for presence of body fluids from mRNA assay data in mixtures. Forensic Sci. Int.: Genetics. 2021;52 doi: 10.1016/j.fsigen.2020.102455. article 102455. [DOI] [PubMed] [Google Scholar]