Table 1.

Descriptions from clinical and progression decision-making, where individual and group decisions have been compared

Explanation of type of bias and/or error [48]	Description and example from clinical decision-making	Description and example from progression decision-making	Effect on decision-making as a group vs as an individual [48]
Framing: decisions vary with the context in which the information is presented.	A patient has had several visits to an ED with a headache, and on each occasion has been diagnosed as having migraine. On this visit, the clinician assumes that the patient has migraine again [15].	Substandard performance at the start of year creates the label of a “poor performing” student, and performance later in year is assumed to represent poor performance [51]. A similar bias for above-standard performance is also possible.	Mixed effects as amplification, attenuation and no change have all been reported
Preference reversal: the judgement outcome depends on how the data are presented	Choices made by clinicians and patients when considering management options will vary dependant on information how information is presented [52]. As an example, the probability of getting a condition whilst taking a drug is 96%, and when not taking the drug is 98%. Put another way, the chance of getting the disease is halved (from 4 to 2%). The prevalence of side effects form the drug is 10%: put another way 90% have no problems at all. The patient decision may alter depending on which figures are provided.	When faced with making a decision for a student at the borderline of satisfactory performance, the outcome may differ if the focus is on being fair to the student or if the focus is on protecting the public.	Mixed: amplified on and attenuated
Theory-perseverance effect: confirmation bias (only looking for information that will support a decision) and ascertainment bias (only finding information that will support a decision).	Confirmation bias occurs when a clinician only seeks out evidence that supports a the proposed diagnosis, such as only asking about cardiac symptoms in a person with breathlessness [15]. Ascertainment bias occurs when a clinician preferentially finds supporting evidence, such as finding evidence of heart failure in a patient with breathlessness who has been noncompliant with diuretic medication [15].	When observing a student, the examiner forms an initial first impression of the student as being below standard, and thereafter only looks for evidence of poor performance/only finds evidence of poor performance. A similar bias for above-standard performance can also occur [53].	Attenuated by group
Weighting sunk costs: continue to invest in a losing transaction because of losses already incurred.	A decision to continue ineffective treatment, such as ongoing treatment for progressive malignancy [54].	A student who has progressed a significant way through a course (e.g. to final year), before their substandard performance comes to attention. They may be harder to fail as “they’ve got this far”, and be given the benefit of the doubt that they are a potentially failing student [36, 55].	Amplified by groups
Extra-evidentiary bias: Irrelevant information influences decision-making.	Clinical decision-making regarding an individual patient may be informed by trial results, which then requires extrapolation by clinicians identifying similarities and differences between the patients in the trial and the individual patient. This process is often influenced by extra-evidentiary considerations, such as personal clinical experience [56]. Additionally, combining information as part of clinical decision-making requires appropriate aggregation rules utilising the tools of mathematics (e.g. set theory, symbolic logic, and Boolean algebra) to support more reliable decisions [56].	A body of assessment evidence suggests a student is passing, but an influential senior staff member provides a single anecdote of substandard performance that sways the decision. This can work both in favour of, and against the student [57].	More amplification than attenuation for groups
Hindsight bias: knowing the outcome alters recollections; assigning inferences; ignoring prevalent circumstances.	When events are viewed in hindsight, there is a strong tendency to attach a coherence, causality, and deterministic logic to them, such that no other outcome could possibly have occurred, thereby distorting the perception of previous decision-making [15].	Most doctors with professional conduct problems in practice had professional conduct problems in medical school [58, 59]. Awareness of this could lead a medical school to erroneously fail any student with professional conduct problems during a medical course, yet the vast majority of students with professional conduct problems become clinicians with no professional conduct lapses.	Attenuated for groups
Insensitivity to base rate (underuse of representative heuristic): frequency within population is ignored in estimating probability	If all causes of pleuritic chest pain are considered to have equal pre-test probabilities, then they are all assumed to have equal prevalence rates. This can lead to over-investigation of less likely causes (e.g. pulmonary embolus) and therefore an overestimation of the post-test likelihood [15].	A single performance just below the standard (e.g. in an end-of-year OSCE) in an assessment with a high pass rate (e.g. > 95%) by a student is given too much weight, when the student has clearly been above standard to date in all equivalent assessments, and the pre-test probability of passing should be high [60].	Mixed results
Overuse of presentative heuristic: overreliance on some salient information; stereotyping based on similarities.	The patient’s symptoms and signs are matched against the clinician’s mental templates for their representativeness. Clinicians base diagnostic decisions about whether or not something belongs to a particular category by how well it matches the characteristics of members of that category [15]. A patient presenting with atypical symptoms and signs, such as a young female with a history of psychiatric disease, can lead to myocardial infarction not being considered, and the patient being sent home from Emergency Department.	A student who is a member of a specific group (e.g. male ethnic minority) that performs less well in assessments [61] is expected to demonstrate lower performance.	Mixed: Amplified by groups or no effect
Overconfidence (miscalibration): belief in the probability of being correct is greater than actual.	Overconfidence by a clinician thinking they know more than they do, leading to gathering insufficient information [15].	A clinician may consider themselves, rightly or wrongly, to be a good clinician, and therefore also assumes they will also be a good assessor. An individual will put greater weight on their decisions than is justified by the evidence [62].	Mixed effects