More skilled, more cautious: Expertise and decision thresholds in fingerprint examination

Abstract

Fingerprint examination relies on expert judgment to determine whether two impressions originate from the same source, yet variability exists across examiners evaluating identical evidence. The present study investigates the sources of this variability by distinguishing between examiner expertise and decision thresholds within a signal detection theory framework. Using data from two large-scale black box studies: the Latent Print Black Box 2 Study [1] and the Kibbey et al. dataset, we estimated examiner-level discriminability (d′) and decision thresholds using an extension of signal detection theory with multiple decision criteria.

Across both datasets, examiner's expertise as operationalized by (d’) was positively associated with identification thresholds in the signal detection theory model, indicating that more skilled examiners adopt more conservative (risk averse) threshold when making same-source conclusions. In contrast, expertise was not associated with exclusion thresholds, suggesting that the threshold for different-source decisions operate independently of perceptual sensitivity. Additionally, we examined whether individual differences in personality predicted decision thresholds and found no statistically significant relationships, suggesting that these traits may exert influence primarily under conditions of ambiguity rather than in standard casework-like comparisons.

This work extends prior signal detection theory findings by demonstrating that a change in discriminability (as measured by d’) is accompanied by differences in decision-threshold placement, such that examiners with greater discriminability also tend to adopt a more conservative Identification threshold.

Highlights

• •Higher expertise is associated with more conservative identification thresholds.
• •No association between expertise and exclusion thresholds was found.
• •Variability exists between examiners evaluating same evidence.
• •Personality effects may depend on ambiguity of the task.
• •Variability arises from both perceptual sensitivity and decision threshold.

1. Introduction

Fingerprint examination is a forensic science discipline that focuses on the analysis and comparison of friction ridge impressions such as fingerprints and palmprints. Examiners recover prints from crime scenes or objects, assess the quality and quantity of specific minutiae such as ridge endings and bifurcations, and compare the prints against known reference prints collected under controlled conditions. Fingerprint examiners communicate their results using various scales to express their conclusions and in the United States, these have taken the form of a definitive conclusion. Historically, the most widely used framework consisted of three conclusions: Exclusion, Inconclusive, and Identification [1]. More recently, laboratories and professional organizations have adopted expanded scales that include additional categories, such as Source Identification. Examiners rely on historical precedent, standardized terminology, their training, their experience, and laboratory SOPs to decide which terms to use. Even when using standardized verbal scales, examiners can differ in how they interpret and apply these categories, leading to variability in decisions across individuals when evaluating the same fingerprint evidence [2], [3]. Work by Luby [4] demonstrated that examiners do not use conclusion categories in a uniform manner, and show variability in the frequency of inconclusive, exclusion, and identification decisions across identical comparisons. Note that other forms of evaluative reporting such as likelihood ratios and probabilistic statements may not involve a threshold, a point we return to at the end of the paper. We and the field tend to use the term Identification and Source Identification interchangeably and do so in this paper. The capitalization of “Identification” is intentional as it refers to the decision made by the examiner.

1.1. Expertise

The first source of variability we will consider is expertise: Broadly, expertise has been defined as the possession of domain‐specific knowledge that is acquired through experience or training and that leads to superior, reproducible performance in domain‐related tasks [5]. This definition highlights the domain-specific nature of expertise and ties it directly to performance. However, the literature on expertise is mixed. In some domains, such as chess, experts consistently outperform novices [6], while in others, such as sports prognostication, expertise does not appear to provide a significant advantage [7].

To be considered competent to perform work by their agency, fingerprint examiners undergo extensive training, though the specifics of their training can vary. Fingerprint examiners are frequently required to take proficiency tests which are assessments designed to evaluate their ability to accurately identify and compare forensic evidence [1]. These tests can be conducted internally or externally, and they can be announced and planned or blind and mixed into casework [8].

Differences between examiners have emerged depending on the level of expertise. Thompson, Tangen and McCarthy [9] revealed that qualified, court-practicing fingerprint experts exhibited exceedingly high accuracy, significantly outperforming novices. Additionally, experts were more conservative in their judgments, setting a higher threshold for identification and thereby reducing false positives. However, despite their superior performance, the study acknowledged that experts are not infallible, and errors can still occur. Building on these findings, Thompson and Tangen [10] extended their investigation to examine whether the nature of expertise in fingerprint examination is primarily driven by an ability to recognize mated pairs or by a more refined capacity to distinguish highly similar but non-matching prints. They found that experts were even more conservative when analyzing genuine crime scene prints compared to simulated prints used in prior studies. While this conservative approach reduced the likelihood of false identifications, it also resulted in more instances where true mated pairs were missed. Nevertheless, experts maintained a high level of accuracy compared to other groups.

One of the reasons fingerprint experts perform better than novices is that expertise changes the way individuals process information. Research suggests that experts tend to look at less information overall but process it more quickly and efficiently. Unlike novices, who may rely on broad, surface-level details, experts depend on specialized, experience-based cues sometimes referred to as “broken leg” cues that allow them to make rapid yet informed judgments [11]. These cues are often subtle and not immediately apparent to those without extensive experience in the field [12]. As a result, their ability to recognize and interpret domain-specific signals gives them a significant advantage over novices.

We will operationalize expertise within the signal detection theory model as discriminability or d’. We discuss this operationalization later when we describe the signal detection theory model.

1.2. Definitiveness/cautiousness

A major source of variation among fingerprint examiners is the definitiveness of different examiners [13]. Trainee examiners often have difficulty “getting off the fence” when Inconclusive is an option, although given their relative lack of expertise such a strategy may be optimal. A challenge with forensic testing such as black box studies is that an examiner with a high Identification rate may be good, cocky, or both [14]. This argument was nicely summarized by Eldridge, De Donno and Champod [14] (p11): “If the majority of examiners reach an inconclusive decision, but some examiners reach a definitive decision (identification or exclusion), were those definitive examiners “super-examiners” or were they simply too risk-tolerant and making decisions that were not sufficiently supported by the available data? Conversely, if an examiner reports inconclusive when the majority reached the correct definitive conclusion (i.e.the conclusion that matches ground truth), was the inconclusive examiner the lone voice of reason to exhibit caution, or were they being too risk-averse?”

We will operationalize the definitiveness/cautiousness of an examiner using the decision thresholds from signal detection theory described below.

1.3. Personality

The third source of variability is personality, which refers to stable patterns of cognition and behavior that distinguish individuals from one another. Personality shapes how people process information and make decisions [15]. In fingerprint examination, these individual differences can influence how examiners engage with the comparison process and how they translate their observations into verbal conclusions. One personality trait that is particularly relevant in this context is the Need for Closure (NFC), defined as an individual's desire for a clear, definite answer and an aversion to ambiguity [16]. Individuals with high NFC often exhibit two key cognitive tendencies. The first, known as “seizing,” describes the tendency to reach a conclusion as quickly as possible. The second, referred to as “freezing,” involves the tendency to rigidly preserve an initial decision for an extended period, resisting reconsideration even when new information becomes available Kruglanski and Webster [17]. In fingerprint examination, where impressions can be ambiguous, NFC has been shown to influence examiners identification threshold which refers to the amount of information needed to make an identification. Indeed, research indicates a positive correlation between an examiner's NFC score and the number of identification conclusions they made, suggesting that those with stronger desires for definite answers were more likely to commit to a conclusive identification rather than using the inconclusive category when faced with ambiguous latent prints [18]. Intentionally ambiguous pairs were selected from the Busey, Klutzke, Nuzzi and Vanderkolk [19] dataset, focusing on comparisons that had high variability in examiner judgments, including both Inconclusive and Identification responses. Other studies have further highlighted how this personality trait pushed examiners to avoid ambiguity and provide decisive conclusions and minimize uncertainty in reporting, especially those related to high-profile crimes [20]. This aligns with the broader psychological understanding of NFC as individuals with a high NFC are more likely to seize upon an initial judgment and maintain that decision, even in complex or ambiguous situations.

An additional personality trait that has been shown to play a role in fingerprint analysis is General Self-Efficacy (GSE) which refers to an individual's generalized belief in their ability to successfully perform tasks and overcome challenges [21]. Higher GSE has been associated with more effective coping strategies, greater problem-solving engagement, and better adaptation to stressors, as well as increased optimism and work satisfaction and decreased anxiety, depression, and burnout [22]. This protective function of GSE is particularly important in high-stress environments with prolonged exposure to stress such as the forensic field [23]. Indeed, examiners with high GSE are likely to approach their work with more confidence in their abilities. Their belief in their skills allows them to trust their judgment, which can lead to more focused thinking, even in high-pressure situations. Additionally, this self-assurance helps them resist external pressures, whether from colleagues, deadlines, or environmental stressors. As a result, examiners who measure higher in the trait tend to have higher identification thresholds when presented with ambiguous evidence, suggesting that greater GSE is associated with less risk taking as examiners require stronger information before committing to an identification [18].

1.4. Signal detection theory

Expertise and preference/bias are traditionally thought of as different concepts, but can be difficult to separate by looking only at Identification decision. Expertise relates to the ability to separate mated from non-mated pairs regardless of the placement of a decision threshold. The decision threshold represents risk aversion/definitiveness on the part of the examiner. For example, an examiner with a high rate of correct Identification decisions may have high expertise and a high decision threshold, or less expertise combined with a risk-seeking (lower) decision threshold. To separate expertise from preference/bias, we can use a theoretical approach known as signal detection theory (SDT) [24], [25]. SDT is a model of the decision-making process that uses performance data combined with ground truth to explicitly separate expertise from preference/bias. In this formulation, we assume an underlying dimension that represents the strength of some signal. In the case of fingerprint comparison, the signal being detected is the “matedness”, and the result of a comparison produces some value along this latent dimension. To make a decision, the examiner will compare that value along the latent dimension against a set of thresholds that represent their preference/bias levels. For example, if the latent value exceeds the highest threshold, the examiner will make an identification decision.

The advantage of SDT is that it explicitly separates expertise from the decision threshold, and these are thought to be independent and separable constructs within the model. Fig. 1 illustrates the structure of SDT. In panel A, the two curves represent the distribution of the latent values for mated and non-mated pairs along the latent axis for a reference examiner. The vertical lines represent the decision thresholds that partition the latent dimension. Expertise is represented by the separation between mated and non-mated curves such that greater distance implies greater expertise. In Panel B of Fig. 1, the examiner has greater expertise. In principle, the relation between the separation of a curve and the position of the decision threshold is independent: You could have two examiners who both possess high expertise, but one is risk averse and the other risk seeking. In this case, the two subjects would have similar mated distributions but different identification thresholds, as shown in Panel C. Examiners with greater expertise might take advantage of that shift by also shifting the identification decision threshold to the right to reduce the chance of an erroneous identification while still contributing evidence that will convict guilty persons [26], [27] as shown in Panel D. This behavior might results from the fact that after conducted a larger number of comparisons over the course of their careers, they may be more aware of the potential consequences of an erroneous identification and therefore less willing to take risks [28]. This might produce a rightward shift in the identification threshold even as the mated distribution shifts to the right. Thus, we might predict a positive correlation across examiners between their expertise and their risk aversion such that the best examiners are also the most risk averse.

Fig. 1.

Illustration of Signal Detection Theory (SDT).The figure shows overlapping distributions for nonmated and mated comparisons along a continuous axis representing support for a common source. Dashed vertical lines represent decision thresholds for five conclusions: exclusion, inconclusive due to dissimilarities, inconclusive, inconclusive due to similarities, and identification. The location of the distributions reflects an examiner's expertise (d′), whereas the placement of the thresholds reflects decision strategy. Panel A illustrates a reference examiner. Panel B illustrates increased expertise, represented by greater separation between the nonmated and mated distributions while holding the decision thresholds constant. Panel C illustrates a more risk taking strategy, represented by shifting the decision thresholds toward lower levels of support for a common source while maintaining the same discriminability as the reference examiner. Panel D illustrates an examiner with both increased expertise and a more cautious decision strategy, represented by greater separation between the distributions combined with thresholds shifted toward higher levels of support for a common source.

Note that greater expertise combined with a more conservative Identification threshold does not necessarily imply fewer correct identifications. The resulting identification rate depends on the relative magnitude of the shift in the mated distribution and the shift in the decision threshold. For example, consider an examiner whose mated distribution has a mean of 1 and a standard deviation of 1, with the non-mated distribution fixed at a mean of 0 and a standard deviation of 1, and whose Identification threshold is located at .5. If the mated mean shifts one standard deviation to the right while the Identification threshold shifts only .5 units to the right, the examiner would be both more expert and more cautious, yet would make more correct identifications than under the original configuration. In contrast, if the mated mean shifts only .5 units to the right while the Identification threshold shifts one full unit to the right, the examiner would again be both more expert and more cautious, but would make fewer correct identifications. Thus, raw correct Identification rates alone cannot distinguish changes in expertise from changes in decision threshold placement. This reinforces the need to decompose the raw responses into d’ and decision thresholds.

In this paper, our goal is to investigate individual differences among examiners to highlight variation and their sources. To investigate these sources of variability, we used data from two black box studies [29]: and the Latent Print Black Box Study 2 [30]. Further descriptions of these datasets are provided in the Method section. Across both black box studies, we analyzed examiners' performance as well as examiners’ expertise. In addition, we examined whether individual differences in NFC and GSE were associated with performance metrics within the Kibbey et al. black-box dataset. Personality data were collected in two waves. For a subset of examiners, NFC and GSE data were already available from participation in an earlier study. We subsequently contacted the remaining examiners from the original Kibbey et al. black-box sample for whom personality data were not yet available and invited them to complete the same measures.

2. Method

All analyses are found at this OSF site:

https://osf.io/vn9x6/overview?view_only=ccac8c6fbc4740b88ee0d559d5344db1.

The archive contains an excel file explaining the analyses. The parameters for the signal detection theory modeling are found in the osf. zip file in the archive, and these parameters are correlated with the personality variables using the files in the main archive as described by the readme excel file.

2.1. Latent Print Black Box 2 study [30]

The Latent Print Black Box 2 Study, reported by Hicklin and colleagues [30], evaluates the accuracy, reproducibility, and variability of latent fingerprint examiner decisions. We chose the Hicklin dataset as it is considered the most recent and comprehensive Black Box study. 156 latent print examiners were given a random selection of 100 latent-exemplar image pairs from a pool of 300 possible pairs with a ratio of 80:20 non-mated: mated. For each comparison, examiners used the following 7-level categorical conclusion scale: Exclusion–Non Borderline (Excl-NB), Exclusion– Borderline (Excl-B), Inconclusive–Dissimilar (Inc-Dis), Inconclusive no similarities/dissimilarities indicated (Inc-No), Inconclusive–Similar (Inc-Sim), Identification–Borderline (ID-B), and Identification– Non Borderline (ID-NB). We defined the Identification Threshold as the SimilaritiesIDBThreshold in the raw data.

2.2. Kibbey et al. [29]

In this black box study, 104 participants completed a series of 60 fingerprint comparisons. Each examiner was asked to conduct a full analysis and give a conclusion using a 5-point response scale, with two brief follow-up questions after each comparison. Participants were given up to one month to complete all trials. The image pairs were evenly divided between mated and non-mated trials and selected to be broadly representative of the range of complexity encountered in casework, rather than to induce errors or mislead examiners. Automated Fingerprint Identification System exemplars were not used to generate the non-mated trials, in contrast to the Hicklin study [30].

Although both black box studies are not equivalent, both were designed to provide challenging non-mated comparisons rather than randomly selected non-mates. The Hicklin study (ref) used a large AFIS search to create close nonmates, which is consistent with operational practices but loses the ability to publish the images. The Kibbey, Coon, Aggadi and Busey [31] study used close non-mates created by using adjacent figures or left-right reflections of the opposite finger on the other hand. Recent work suggests that different fingers from the same individual can share substantial structural similarity. Guo, Ray, Izydorczak, Goldfeder, Lipson and Xu [32] using a deep contrastive learning approach, found strong similarities across fingerprints from different fingers of the same individual, with ridge orientation and curvature, particularly around the central region of the fingerprint, contributing substantially to this cross-finger similarity. This creates challenging close non-mates, although the latent images in the Kibbey et al. study tended to be higher quality than the Hicklin study.

2.3. Personality data

Examiners from the original Kibbey et al. black-box sample for whom personality data were not already available were invited to complete the personality survey. The survey included the Need for Cognitive Closure Scale [16], a 41-item scale with five subscales: preference for order, preference for predictability, decisiveness, avoidance of ambiguity, and closed-mindedness. Participants indicated their agreement on a six-point Likert scale from definitely disagree to definitely agree (Cronbach's α = .80, M = 3.53, SD = .66, range = 1.58–5.25). Then participants completed the General-Self-Efficacy scale with instructions specifying to think about the questions in relation to their workplace [21]. The GSE survey have reported Cronbach's alpha values ranging from .76 to .90 across samples from 23 countries. Items were rated on a seven-point Likert scale from Completely Disagree (1) to Completely Agree (7). Additionally, examiners responded to statements such as “My laboratory's procedures regarding false positives are overly punitive,” “I worry that making mistakes could harm my professional reputation,” and “I fear losing my job if I make an error.” Responses were recorded on a five-point frequency scale ranging from Never to Always.

2.4. Parameter estimation

Signal detection theory decomposes response distributions from individual examiners into estimates of the underlying mated and non-mated distributions as well as 4 decision thresholds for each examiner as illustrated in Fig. 1. To estimate these parameters we used hierarchical Bayesian parameter estimation [33] to estimate the mean and standard deviation of the mated distribution for each participant. To fix the scale, we set the mean of the nonmated distribution to 0 and the standard deviation to 1.0. This allowed us to freely estimate the four decision thresholds. Each subject has 10 datapoints and 8 degrees of freedom. We fit six free parameters (mu, sigma, and four thresholds) for each subject, so the model is not saturated. We used diffuse priors on the mu values and explore even more diffuse priors in the SI. We did not include shrinkage on the standard deviations of the mated distributions because this is an individual differences study. The SI contains a sensitivity analysis where we included shrinkage on the standard deviations of the mated normal distribution. However, we found that this introduced a correlation between d’ and the identification threshold in our simluated data that were created specifically without a correlation between the two parameters. For this reason we did not include shrinkage for the model fits we discuss in the main paper.

Following conventional signal detection theory parameterization, all four decision thresholds were measured along the latent evidence axis defined by the non-mated distribution for which the mean and standard deviation were fixed at 0 and 1, respectively. The location of the Identification threshold is interpreted relative to the non-mated distribution. We recognize that the characterization of an examiner as more or less cautious can depend on the frame of reference used to define the threshold. However, defining one model parameter relative to another estimated parameter would introduce interpretational difficulties. We therefore kept the conventional SDT parameterization and interpreted greater values of the Identification threshold as greater caution within this specific model: that is, more evidence along the standardized latent axis is required before an examiner gives an Identification response, measured relative to the nonmated distribution. Importantly, this interpretation is model-based, and conclusions regarding relative caution should therefore be understood in the context of the parameterization and assumptions of the present model.

2.5. Data analysis

Data analysis on Kibbey et al. and the Latent Print Black Box 2 Study datasets produced examiner-level parameter estimates that separate discriminability from decision threshold. Within this framework, d′ is a measure of examiner expertise, representing the ability to distinguish between mated and nonmated comparisons. We computed d’ by dividing the mated mu value by the square root of (1+ sigma²)/2) where sigma is the standard deviation of the mated distribution. The model estimates decision thresholds that define the boundaries between exclusion, inconclusive, and identification conclusions, quantifying how much support an examiner requires to move between response categories. We operationalized risk aversion as the location of the Identification Threshold.

There is one complication with signal detection theory as applied to data from fingerprint examiners. For a varity of (probably reasonable) reasons, examiners adopt a fairly risk-averse identification threshold. This means that they will have very few, if any, erroneous identification outcomes. This means that the rest of the responses will have to constrain the location of the identification threshold. However, if examiners make only identification decisions to mated pairs, the placement of the mated distribution is only constrained by the prior placed on mu. In this case, the model attempts to push the mated evidence distributions infinitely to the right, while simultaneously shifting the Identification threshold to the left to accommodate the absence of errors. This leads to unstable and non-interpretable parameter estimates and underspecified thresholds. In addition, less than perfect but very high accuracy will also lead to issues with parameter estimation. We examined the posterior uncertainty associated with extreme d′ estimates by calculating the lower and upper bounds of the highest density interval (HDI) for each subject. We found that when d′ exceeded 4, the associated HDIs became extremely large (going from approximately 8 to over 100), indicating parameter uncertainty. This inflation of uncertainty reflects a known limitation of signal detection theory when applied to perfect subjects, for whom minimal variability in errors leads to disproportionate changes in d′ estimates. 23 examiners in the Kibbey, Coon, Aggadi and Busey [29] data and 11 examiners in the Black Box 2 Study [30] exhibited extremely high sensitivity estimates (d′ > 4) with very high uncertainty on the parameter estimates. These subjects were pruned from the analysis. This is a fundamental limitation of signal detection theory (and any test where subjects perform perfectly) and below we discuss implications of excluding these subjects because they are also our very best subjects and we focus parts of our analyses on expertise.

For the remainder of the participants (81 participants in the Kibbey et al. data, and 145 participants in the Black Box 2 Study), the present analyses focused specifically on the Identification threshold and the Exclusion threshold, as these parameters capture the threshold required to reach definitive same-source and different-source conclusions, respectively. These examiner-level estimates were then correlated with personality measures collected via post-task surveys to examine how individual differences relate to expertise for 46 participants. One Black Box 2 examiner accounted for 13 of the 23 erroneous identifications; however, a sensitivity analysis excluding this examiner showed that the correlation between discriminability and the Identification threshold remained statistically significant, although it decreased from r = .39 in the full sample to r = .28.

The OSF link contains a readme file that further describes all the files and analyses performed. We invite the reader to run our codes and generate our analyses.

3. Results

Fig. 2 displays the relative likelihood of observing a given latent evidence value for individual examiners in the Latent Print Black Box 2 Study, with each distribution defined by its mean (μ) and standard deviation (σ) at the examiner level. Each blue curve represents the latent evidence distribution for an individual examiner, illustrating how strongly that examiner tends to generate same-source conclusions across comparisons. The red curve represents a fixed reference distribution corresponding to different-source evidence and is held constant across examiners. Differences in the horizontal placement of the blue curves along the latent evidence axis reflect variability in examiner sensitivity: blue curves that are shifted further to the right indicate greater separation from the fixed reference distribution, corresponding to larger d′ values and higher examiner expertise. The vertical height of each curve represents the relative density of the normal distribution, reflecting the relative likelihood that a given examiner would produce a particular latent evidence value for a mated or non-mated pair. Together, these distributions illustrate how examiners vary in their ability to discriminate between same-source and different-source comparisons.

Fig. 2.

Relative likelihood of observing a given latent evidence value for individual examiners in the Latent Print Black Box 2 Study. Each blue curve represents the estimated latent evidence distribution for a single examiner, with parameters derived from the ordered probit model fit to that examiner's categorical decisions. The red curve represents a fixed reference distribution corresponding to nonmated comparisons. Examiner expertise is reflected in the horizontal separation between the blue and red curves: blue curves shifted further to the right indicate greater separation from the fixed nonmated distribution, corresponding to larger d′ values and therefore higher discriminative ability.

The left side of Fig. 3 illustrates the relationship between examiner expertise, d′, and the Exclusion threshold in the Latent Print Black Box 2 Study. Each point represents an individual examiner, reflecting how much evidential support that examiner required to give an exclusion decision. The fitted least-squares regression line shows no association between both variables, and the correlation analysis revealed no association between expertise and exclusion threshold (r = .13, p = .107, n = 145, df = 143), suggesting that the ability to discriminate between mated and nonmated comparisons is independent of examiners’ exclusion thresholds. Conversely, the right side of Fig. 3 shows the relationship between examiner expertise, d′, and the Identification threshold in the Latent Print Black Box 2 Study and reveals that examiners with higher expertise tend to adopt more conservative (risk averse) identification thresholds. The correlation analysis indicates a small but statistically significant positive association between expertise and identification threshold (r = .39, p < .001, n = 145, df = 143), suggesting that examiners with higher expertise require stronger internal evidence before giving a definitive identification. Additionally, we performed Spearman correlations and found the following results: A positive correlation between expertise and identification threshold (r = .262, p < .001, n = 81, df = 79) as well as no correlation between expertise and exclusion threshold (r = .136, p = .119, n = 81, df = 79).

Fig. 3.

Left side: Relationship between examiner expertise (d′) and the Exclusion threshold in the Latent Print Black Box 2 Study. Each point represents an individual examiner, with the solid blue line indicating the least-squares regression fit. The analysis revealed no association between the variables (r = .08, p = .39). Right side: Relationship between examiner expertise (d′) and the Identification threshold in the Latent Print Black Box 2 Study. Each point represents an individual examiner. The solid blue line shows the least-squares regression fit,. Results indicate a small but statistically significant positive association between the variables (r = .32, p < .001).

To examine whether this pattern was specific to the use of d′ as the measure of examiner expertise, we conducted an additional analysis using AUC probit as an alternative measure of discriminative ability. Consistent with the analyses based on d′, AUC probit was positively associated with identification thresholds (r = .3, p < .001), indicating that examiners with greater discriminative ability tended to require stronger evidence before making an identification. In contrast, AUC probit was not significantly associated with exclusion thresholds (r = .14, p = .103), again suggesting that greater discriminative ability was specifically related to the placement of the identification threshold rather than to a general tendency to adopt more conservative decision criteria.

We performed similar analyses on the Kibbey et al. Datasets with Fig. 4 illustrating the relative likelihood of observing a given latent evidence value for individual examiners. Each curve represents the latent evidence distribution estimated for a single examiner, with horizontal shifts along the latent axis reflecting differences in examiner expertise. As in the Latent Print Black Box 2 Study, examiners whose distributions are positioned further to the right have higher d′ values and greater expertise. The height of each curve represents the probability density of observing a particular latent evidence value for that examiner. Building on these distributions, The left side of Fig. 5 illustrates how examiner expertise is associated with Exclusion threshold for each examiner. Each point represents an individual examiner. Similar to the Latent Print Black Box 2 Study, no association was observed between expertise and exclusion threshold (r = .02, p = .859, n = 81, df = 79), indicating that examiners with greater expertise don't predictably apply conservative or aggressive exclusion threshold. As in Fig. 3, the right side of Fig. 5 shows a clear positive association between expertise as measured by d’ and Identification thresholds, indicating that examiners with higher discriminative ability have more conservative identification thresholds. This significant relationship (r = .42, p < .001, n = 81, df = 79), suggests that greater expertise is associated with requiring stronger evidential support before committing to an identification decision. Again, we performed Spearman correlations and found a positive correlation between expertise and identification threshold (r = .43, p < .001, n = 81, df = 79) as well as no correlation between expertise and exclusion threshold (r = .01, p = .89, n = 81, df = 79).

Fig. 4.

Relative likelihood of observing a given latent evidence value for individual examiners in the Kibbey et al Datasets. Each blue curve represents the estimated latent evidence distribution for a single examiner, with parameters derived from the ordered probit model fit to that examiner's categorical decisions. The red curve represents a fixed reference distribution corresponding to nonmated comparisons. Examiner expertise is reflected in the horizontal separation between the blue and red curves: blue curves shifted further to the right indicate greater separation from the fixed nonmated distribution, corresponding to larger d′ values and higher discriminative ability.

Fig. 5.

Left side: Relationship between examiner expertise (d′) and the Exclusion threshold in the Kibbey et al. Datasets. Each point represents an individual examiner, with the solid blue line indicating the least-squares regression fit. The analysis revealed no association between the variables (r = 0.02, p = .86). Right side: Relationship between examiner expertise (d′) and the Identification threshold in the Kibbey et al Datasets.Each point represents an individual examiner. The solid blue line shows the least-squares regression fit,. Results show a strong positive association between the variables (r = .42, p < .001), indicating that higher values of expertise are strongly associated with greater risk aversion.

Similarly to the Latent Print Black Box 2 Study, we conducted an additional analysis using AUC probit as an alternative measure of discriminative ability. The AUC probit analysis showed a similar pattern. Higher AUC probit scores were associated with higher identification thresholds (r = .28, p = .01), suggesting that better-performing examiners set a more stringent criterion for identification decisions. By comparison, there was no association between AUC probit and exclusion thresholds (r = −.05, p = .651). Together, these results provide converging evidence that the observed relationship between expertise and identification thresholds is not dependent on a single operationalization of examiner performance. Figures associated with the correlation between the thresholds and the AUC probit can be found in the Supplemental Material.

3.1. Correlations with personality

We examined whether individual difference measures were associated with examiners’ decision thresholds for exclusion and identification. Across all analyses, no reliable relationships were observed between personality measures and threshold placement.

As shown in Fig. 6, NFC was not significantly related to either exclusion thresholds or identification thresholds. The association between NFC and exclusion thresholds was small and non-significant (r = .14, p = .37, n = 45, df = 43), as was the association between NFC and identification thresholds (r = −.08, p = .59, n = 45, df = 43). Similarly, Fig. 7 shows no evidence that GSE was associated with decision thresholds. Correlations between GSE and exclusion thresholds (r = −.02, p = .89, n = 45, df = 43) and between GSE and identification thresholds (r = −.13, p = .41, n = 45, df = 43) were both non-significant.

Fig. 6.

Relationships between Need for Closure (NFC) and decision thresholds. Scatterplots show associations between NFC and exclusion thresholds (left) and identification thresholds (right). Each point represents a single examiner. The blue lines indicate least-squares regression fits, with grey regions representing 95% confidence intervals. Pearson correlation coefficients (r), p-values, and sample sizes (n) are reported within each panel. No significant relationships were observed between NFC and either exclusion or identification thresholds.

Fig. 7.

Relationships between General Self Efficacy (GSE) and decision thresholds. Scatterplots show associations between NFC and exclusion thresholds (left) and identification thresholds (right). Each point represents a single examiner. The blue lines indicate least-squares regression fits, with grey regions representing 95% confidence intervals. Pearson correlation coefficients (r), p-values, and sample sizes (n) are reported within each panel. No significant relationships were observed between GSE and either exclusion or identification thresholds.

Taken together, these findings suggest that individual differences in NFC, and GSE, are not associated with the placement of exclusion or identification thresholds in fingerprint examination.

Is the correlation between expertise and risk aversion an artifact?

As examiners make more correct Identifications, the model can either move the mated distribution to the right or sliding the Identification threshold to the left. Normally, the threshold is further constrained by the number of erroneous Identification, but there are none in our data and very few in the Noblis data. Because of this, trade-offs can emerge between the mean (mu) and the identification threshold despite the fact that in principle d’ and thresholds are independent in single detection theory. We worried that apparent correlations between these variables might be an artifact of the signal detection theory model in terms of how the parameters are estimated. To evaluate this possibility, we conducted simulation analyses using both the Kibbey et al. Datasets and the Latent Print Black Box 2 Study dataset. In these simulations, we took the output of the models (mated means and thresholds) and then randomly reassigned all the thresholds for that subject to a different subject to break any true underlying associations between d’ and thresholds. We then used this new set of parameters to generate simulated data with the same counts as in the actual experiment and re-estimated the parameters using our Bayesian model to test for a correlation between d’ and the identification threshold. As described in the Supplemental Material, the simulated data failed to reproduce the correlations observed in our data between d’ and the identification threshold. This suggest that the original associations are unlikely to be a result of an artifact of our parameter estimation methods.

4. Discussion

Our study investigated the sources of variability in fingerprint examiner decision-making. We were able to separate examiners' expertise from their decision thresholds, and found in both datasets, examiners’ expertise was not associated with the exclusion threshold, indicating that more skilled examiners were not systematically more or less conservative when giving exclusion decisions. This finding suggests that exclusion threshold might operate independently of perceptual sensitivity. This interpretation is consistent with findings reported by Ulery, Hicklin, Buscaglia and Roberts [3] who showed that exclusion decisions exhibited patterns of variability that were not simply the inverse of identification decisions, and examiner accuracy on mated comparisons did not straightforwardly predict behavior on nonmated comparisons. In contrast, identification thresholds showed a positive association with expertise, particularly in the Kibbey et al. dataset, where higher d′ values were strongly associated with more risk-averse identification threshold. This pattern suggests that expert examiners are not simply better at detecting same-source evidence, but also more cautious in deciding when the evidence is sufficient to be categorized as an identification. This pattern aligns with existing literature highlighting that improvements in perceptual sensitivity are often accompanied by shifts in decision threshold, particularly in high-stakes domains. Within a signal detection theory framework, sensitivity (d′) reflects the degree to which an observer can discriminate signal from noise, whilst decision threshold determine how much evidence is required to commit to a particular response. Previous work has shown that expertise can influence both. Indeed, as sensitivity increases, decision-makers develop more precise internal representation of evidence strength, which in turn allows them to place decision threshold more deliberately. In high-consequence contexts such as fingerprint comparison, this phenomenon manifests as more risk-averse identification thresholds. Work in recognition memory and applied decision-making has shown that skilled observers not only separate signal and noise more effectively, but also strategically adjust their threshold to avoid erroneous identifications [26]. This pattern has also been demonstrated in fingerprint examination. Using a signal detection paradigm with highly similar latent print comparisons and known ground truth, Thompson, Tangen and McCarthy [9] found that qualified fingerprint examiners substantially outperformed novices while adopting more conservative decision strategies. Importantly, the authors argued that expertise in fingerprint examination should be evaluated using signal detection methods because these approaches distinguish true discriminative ability from response strategy, rather than relying solely on overall error rates or accuracy. The results of the current study are consistent with the findings from Thompson, Tangen and McCarthy [9] and suggest that expertise in fingerprint examination may involve greater discriminative ability and a more risk averse decision criterion.

Note that due to degeneracies in parameter estimation with subjects that are perfect or near-perfect, we excluded 23 subjects from the Kibbey et al. study and 11 subjects from the Black Box 2 Study. The fundamental issue we face with these data is that some examiners are very good with respect to the image pairs they were assigned, and they also adopt a very conservative Identification threshold that is entirely consistent with US ethical and moral standards. This means almost no erroneous identifications and relatively few erroneous exclusions for some examiners. For these near-perfect examiners, we cannot reliably estimate the mean of the mated distribution (and therefore d’) AND we cannot reliably estimate the Identification threshold. There is nothing in the data that will constrain either parameter. As the reviewer correct points out, these values are almost entirely determined by the prior assumptions. A ranked correlation cannot recover the parameters, because while we can assume that the mated mean (and therefore d’) are both high for perfect subjects (and therefore have the highest rank), we can make no such assumption about the location of the identification threshold. It is not identifiable except for being higher than the exclusion threshold (which for perfect subjects is also not identifiable). Because of this lack of identifiability of the parameters for perfect and near-perfect examiners, the correlations (even spearman correlations) done on the entire dataset are uninterpretable rather than contradicting the main analysis. It is possible that a black box study that included even more challenging close non-mates might provide better parameter estimates. Also recall that in these two studies the examiners can declare a latent print to be not “of value” and not have to complete a comparison. If an examiner chooses to set an extremely high threshold for what they deem “of value” for comparison, they effectively eliminate the more challenging prints. Thus an examiner can be perfect simply by choosing to only compare the easiest prints, which is equivalent to taking a test and only answering the easiest questions. We (and the field) have no easy way to compensate for different “of value” thresholds across examiners, but this may be one explanation for the high and perfect accuracy of some of our subjects. However, when we correlate the number of no-value determinations against d’, we found only a correlation of .14 which is not statistically significant. Thus our best examiners do not seem to have adopted a high threshold for “of value”.

It is interesting to note the range of d’ values found in both datasets we analyzed. Indeed, the Kibbey et al. Datasets had d’ values ranging from 1.41 to greater than 4 and the Latent Print Black Box 2 Study had d’ values ranging from .92 to greater than 4. These broad ranges indicate variability in examiner discriminability and reflect individual differences in perceptual sensitivity and expertise. This finding is consistent with Black Box results that have shown how examiners do not only vary in where they place their decision threshold, but also in the strength and consistency of the information they extract from the same visual material. For example, there is significant between-examiner variability in outcomes for the same comparisons, including differences in suitability determinations and disagreement patterns that persist even under standardized testing conditions [2], [3]. Additionally, work in visual comparison supports the idea that there are stable individual differences in comparison ability that can contribute to wide performance ranges even among trained practitioners, offering a mechanism for why d′ can vary an order of magnitude across examiners [34].

Although part of the observed variability can be attributed to stable individual differences across examiners such as expertise, other sources of variability can be stimulus-driven, particularly when the materials vary in ambiguity and thus differentially engage cognitive and affective traits. The absence of significant relationships between personality traits and decision thresholds in the present study should not be interpreted as evidence that such traits are irrelevant to forensic decision-making. Rather, the current findings suggest that the influence of personality factors may be conditional on task characteristics, particularly stimulus ambiguity. Prior work indicates that individual differences in motivational and affective traits are most likely to shape judgment when evidence is weak, conflicting, or difficult to interpret [17], [35]. In contrast to earlier studies employing more ambiguous or challenging comparisons [18], the stimuli used in the Kibbey et al. study were image pairs intended to be similar to casework, which may have limited the extent to which traits such as Need for Closure were engaged. This interpretation is consistent with theoretical accounts proposing that Need for Closure has its strongest effects under conditions of uncertainty, time pressure, or cognitive conflict, when decision-makers are motivated to reduce ambiguity through rapid closure or decisional rigidity [36]. Similarly, research on self-efficacy suggests that their effects on performance and decision strategies are amplified in demanding or uncertain environments but attenuated when task demands are low and evidence is clear [37], [38].

5. Conclusions

The present work found that within a SDT framework, greater discriminability was associated with more risk averse responding as operationalized by d’ and the Identification threshold. The benefits of SDT is that it separates discriminability and risk aversion into separate parameters. Because these are independent, there is no expectation of association between the two parameters. Instead, we find that the best experts are the most cautious.

The present results might be surprising: The best examiners are also the most cautious. One might think that as an examiner gains experience, they might leave their decision threshold in place to preserve a fixed (low) chance of an erroneous identification, while making more Identification decisions to potentially solve more crime. Instead, over time examiners may also encounter scary close non-mates that make them adjust their decision threshold to become more risk-averse. This might underlie the shift to risk-aversion.

It is important to note that our interpretation of the observed positive association between discriminability and the Identification threshold as reflecting greater cautiousness is model-dependent. Within the present parameterization, a threshold located farther to the right indicates that greater evidence along the latent dimension is required before an examiner makes an Identification decision, which we characterize as more cautious responding. Other conceptualizations of cautiousness could incorporate additional aspects of examiner behavior that are not represented by this threshold parameter. Thus, our conclusion that more discriminating examiners are also more cautious should be understood specifically within the SDT framework and parameterization used here, rather than as a general characterization of cautiousness across all aspects of forensic decision making.

Where is the optimal location of the decision criterion? Clearly, examiners must make some Identification decisions, or no crime is ever solved using fingerprint evidence. The field must consider not only the errors that might occur (and the consequences of those errors) but also the prior probability of a mated pair and how the consumers of their testimony interpret the terms. Another possibility is to abandon decision threshold (and decisions) altogether and adopt a strength-of-evidence approach. This approach is described in a companion paper [39], where we transform the ordered probit parameters into likelihoods that represent the strength of the evidence in the absence of verbal terms or decision thresholds. How to integrate all these findings into improvements to the discipline will require continuing conversations with laboratories and practitioners.

Some readers may find the variation in thresholds among examiners concerning because this implies a different standard of proof depending on who completes the examination. For this reason, a growing movement has called for a move away from categorical conclusions to evaluative reporting, in which the strength of the evidence is expressed probabilistically, typically using likelihood ratios that quantify how much more likely the observed fingerprint evidence is under the same-source hypothesis than under the different-source hypothesis [40]. We concur with this sentiment and view our results as further support for such a shift.

CRediT authorship contribution statement

N. Aggadi: Writing – original draft, Resources, Formal analysis, Conceptualization. M. Coon: Writing – review & editing, Resources, Formal analysis, Conceptualization. G. Kibbey: Writing – review & editing, Resources, Formal analysis, Conceptualization. T. Busey: Writing – review & editing, Resources, Formal analysis, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article: