Identification of acutely sick people: individual differences and social information use

Can humans discriminate between healthy and sick individuals, and if so by what cues? In order to study these important questions, in a recent study, Axelsson et al. let 62 untrained raters decide for 32 facial photos whether the person in the photo was healthy or sick [1]. Photos were taken of 16 healthy volunteers, injected once with a lipopolysaccharide injection (Escherichia coli endotoxin) and once with a placebo injection (with a three to four week interval in between). Photos were taken 2 h after injection. While the authors used this ingenious study design to estimate the average discrimination ability of raters, their analysis did not address two keys aspects pertaining to their research questions. First, they did not report whether and how individuals differ in discrimination ability. Individuals may differ in their ability to discriminate between healthy and sick individuals, and in how they balance the trade-off between sensitivity (i.e. the frequency of sick individuals classified as sick) and specificity (i.e. the frequency of healthy individuals classified as healthy), a.k.a. response bias [2]. Importantly, while the authors did not use their data to investigate the presence and structure of such differences, the implications of their work for disease dynamics—and consequently prevention strategies—will differ substantially between populations with and without individual differences [3–5]. For example, groups and populations with a large number of expert and/or ‘cautious’ individuals (i.e. individuals with a relatively low response bias) are predicted to have a lower prevalence and weaker infectious outbreaks than groups with fewer expert and/or cautious individuals. Second, while the authors investigated the discrimination ability of socially isolated raters, in many real-world contexts, individuals will be able to observe the responses of others to potentially sick individuals. In such situations, the social information present in the responses of others can be used to improve decisions. Thus, while the authors of the original study focused on facial cues of sickness, they did not explore the potential role of social cues provided by other raters. Here, we investigate both issues using the original dataset from Axelsson et al. [1,6].

To investigate whether raters differed in their ability to discriminate between healthy and sick individuals, we determined the average true positive rate (0.533) and the average false positive rate (0.305) in the population. We note that—in the original study—raters occasionally rated the same image twice, in which case we only considered the raters' first decision. We then simulated 10 000 decision-makers with the same true and false positive rates (i.e. corresponding to the average true and false positive rates) evaluating 16 healthy and 16 diseased cases each and calculated the discrimination ability of each simulated decision-maker: d′ = z(hit rate) – z(false alarm rate) [7]. A d′ of 0 indicates no discrimination ability, and higher values indicate a better ability to discriminate healthy from sick. Hit/false alarm rates of 0/1 were converted to 0.01/0.99 to be able to calculate d′. We then determined the 95% confidence interval (CI) of the expected distribution and compared both the distribution and its 95% CI (figure 1c) to the observed distribution (figure 1a). While no consistent differences between decision-makers would give rise to a symmetric distribution with few raters below and above the 95% CI (figure 1c), we observe a right-skewed distribution (figure 1a) with a higher number of high-performers than expected. Four individuals have a performance above the expected 95% CI—whereas the expected number of performers above the 95% CI, based on 62 raters, was 62/100 × 2.5 ∼ 1.6.

Figure 1.

The observed (a, b) and (assuming no individual differences) expected (c, d) distributions of raters' discrimination ability (i.e. d-prime) and response bias when discriminating between healthy and sick individuals (see main text for details). The black and red dashed lines show the median performance and the 95% CIs of the expected distribution, respectively. (a) The observed distribution of raters’ d-prime had a stronger right-skewed distribution than expected (c), indicating a substantially higher number of high-performers than expected. (b) The observed distribution of raters' response bias was substantially more fat-tailed than expected (d), indicating substantial individual differences in how individuals balance the trade-off between sensitivity and specificity. (Online version in colour.)

To further study the presence/absence of consistent individual differences in discrimination ability, we investigated the correlation between raters' discrimination ability when splitting the full set of 32 images randomly in two halves of 16 images. Within each half, we calculated the mean discrimination ability of each rater and we then tested whether there was a correlation between the mean discrimination ability of raters in the first and second half, with the null hypothesis being that there is no relationship between raters’ discrimination ability in the first and the second half (i.e. no consistent individual differences in discrimination ability). Performing this procedure 1 000 times (i.e. for 1 000 different random splits of the 32 images), we find a weak correlation between the discrimination ability of raters in the first and second half (average Pearson's r over 1 000 random splits = 0.15; only 181 out of 1 000 splits p < 0.05), indicating that—while we find a higher number of high-performers than expected—on average we only find weak consistent individual differences in discrimination ability.

Next to discrimination ability, individuals may differ in how they balance the trade-off between sensitivity and specificity, a.k.a. response bias [2]. Analogous to the above analysis, we calculated the response bias of each of the 10 000 simulated decision-makers: [7]. A c-value of 0 implies no response bias, whereas more negative (positive) c-values imply a higher likelihood to classify a given case as sick (healthy). Comparing the expected distribution and its 95% CI (figure 1d) to the observed distribution (figure 1b), we conclude that the observed distribution is substantially more fat-tailed than the expected distribution: 11 (8) out of the 62 individuals had a response bias lower (higher) than the expected 95% CI, suggesting substantial individual differences in response bias. In line with this, following the same procedure as described above, when splitting the 32 images randomly in two halves, we find a positive correlation between the response biases of individuals in the first and second half (average Pearson's r = 0.70; 1 000 out of 1 000 splits p < 0.05). Note that discrimination ability and response bias did not correlate across participants (Pearson's r = −0.02, p = 0.87).

In sum, we find (i) a higher than expected number of expert individuals with a high ability to discriminate between healthy and sick individuals and (ii) substantial differences in how individuals balance the trade-off between sensitivity and specificity. Future studies should investigate the (in)consistency of individual differences over longer time periods and the causes of the individual differences we found. Do expert individuals, for example, have more previous experience with sick individuals, or generally higher cognitive abilities than non-experts? Do differences in cautiousness reflect an adaptive response to differences in the costs and benefits associated with behavioural immunity? Less cautious individuals, for example, may be physiologically, psychologically, or socially less vulnerable than more cautious individuals (see also [8]); and the level of cautiousness may be a response to the frequency of expected sick people in the target population and/or the context (e.g. whether there is a contagious disease outbreak or not) (see also [9]). Furthermore, as certain individuals (e.g. ‘superspreaders’ [4]) play particularly important roles in disease dynamics, it will be important to ask whether the individual differences we find are related to differences in social network position. Networks where the cautiousness of individuals is positively correlated with the number of social contacts, for example, are predicted to have a higher behavioural immunity than networks where these two aspects are not or even negatively correlated.

This brings us to the second issue: the potential role of social cues provided by other raters. Using the data from Axelsson et al., we investigated whether and how an individual in a group can use the decisions of others when discriminating between healthy and sick individuals. In particular, for different group sizes, we simulated the performance of an individual employing either of two basic and well-known rules for social information use: (i) the follow-the-majority rule, where an individual adopts the decision of the majority of the other group members and (ii) the follow-the-best-member rule, where an individual adopts the decisions of the group member that performed best in the past [10,11].

To test the performance of an individual employing either of the two rules when discriminating between healthy and sick individuals in groups of different sizes, we randomly drew virtual groups of n individuals (1–15, only using odd group sizes) from the pool of 62 raters. For each group, we randomly split all 32 images into a training (26 images) and a test set (six images). We used the training set for estimating the identity of the best rater (i.e. the individual with the highest percentage correct) and the test set for testing the performance of an individual employing either of the two rules. We used such a cross-validation procedure because the best member needs to be identified before it can be followed. The follow-the-majority rule does not require such a validation procedure because it implements a fixed rule. However, to make the results between both rules comparable, we tested the performance of both rules only in the test set. The follow-the-best-member rule was implemented as following the decision of the group member with the highest percentage correct in the training set. The follow-the-majority rule was implemented as following the decision of the majority of individuals. We repeated this procedure 2 500 times for each group size (i.e. for 2 500 different random splits of the images in a training and a test set). Figure 2 shows the results of this analysis. In line with the finding above that a small number of individuals are substantially better than others in discriminating between healthy and sick individuals (figure 1a), we find that an individual that employs the follow-the-best-member rule can increase both sensitivity and specificity with increasing group size (generalized linear model (GLM) with binomial link function: effect of group size on sensitivity: z = 12.70, p < 0.001; specificity: z = 5.61, p < 0.001; figure 2a). An individual adopting the follow-the-majority rule can increase specificity but not sensitivity with increasing group size (GLM, sensitivity: z = −1.28, p = 0.20; specificity z = 2.15, p = 0.03; figure 2b). The follow-the-majority rule is known to perform well when the average individual accuracy is well above 0.5 [12], explaining why—for an individual using the follow-the-majority rule—specificity but not sensitivity increased with increasing group size (average individual specificity and sensitivity is 0.70 and 0.53, respectively). In sum, while both rules allow individuals to increase performance when discriminating between healthy and sick individuals, the rule that is based on expert decision-making clearly outperforms the rule that is based on pooling independent decisions (GLM; comparing overall accuracy between the follow-the-best-member rule and the follow-the-majority rule: z = −7.67, p < 0.001). As the potential for exploiting the decisions of others crucially depends on the ability to detect high-performers and to successfully integrate their social information [13], it will be interesting for future research to investigate whether, to what extent, and how individuals rely on social information when being confronted with potentially infectious others.

Figure 2.

The performance of individuals employing the (a) follow-the-best-member rule and (b) follow-the-majority rule. (a) With increasing group size, both the specificity (solid red line) and sensitivity (dotted blue line) of an individual employing the follow-the-best-member rule increased. (b) For an individual employing the follow-the-majority rule, with increasing group size, the specificity increased but not the sensitivity. Results are based on 2 500 randomly drawn groups (see main text for details). (Online version in colour.)

To conclude—by focusing attention on individual differences and social information use—our goal was to complement the recent study by Axelsson et al. [1] that has investigated whether and by which cues individuals can discriminate between healthy and sick people. We believe that, together with the findings of the original study, our results outline key aspects of an exciting and highly relevant research area: the causes and consequences of behavioural immunity in individuals, groups, and populations.

Data accessibility

Data of the original study can be found at https://osf.io/btc7p/. The code of the analyses is included as electronic supplementary material.

Authors' contributions

R.H.J.M.K. and M.W. conceived the presented idea and wrote the paper; R.H.J.M.K. performed the analysis.

Competing interests

We declare we have no competing interests.

Funding

We received no funding for this study.