Table 1. Studies included in the meta-analyses by decision domain and decision-maker expertise.

Study		Judges	Number of judgments	Number of cues	Judgment task	Criterion	Task results
a)	Medical science, experts:
1)	Nystedt & Magnusson [23]	4 clinical psychologists	38	3	Judge patients based on patient	Rating on three	I: Δ1 = .11
					protocols:	psychological tests (■)	II: Δ2 = .03
					I: intelligence		II: Δ3 = .12
					II: ability to establish contact		(*, +, s)
					III: control of affect and impulses
2)	Levi [24]	9 nuclear medicine	280	5	Assess probability of significant	Coronary angiography	Δ4 = .07
		physicians	(60 replications)		coronary artery disease based on patient		(*, s)
					profiles
3)	LaDuca, Engel, & Chovan [25]	13 physicians	30	5	Judge the degree of severity	A single physician’s	Δ5 = .08
					(congestive heart failure) based on	judgment (▲)	(*, s)
					patient profiles
4)	Smith, Gilhooly, & Walker [26]	40 general practitioners	20	8	Decision to prescribe an antidepressant	Guideline expert (▲)	Δ6 = -.05
					based on patient profile		(s)
5a)	Einhorn [27] (This publication	3 pathologists	III: 193	9	Evaluate the severity of Hodgkin’s	Actual number of	III: Δ7 = -.01
	contains two studies)				disease based on biopsy slides	months of survival	(s)
	Second study
6a)	Grebstein [28]	10 clinical experts	30 profiles	10	Judge Wechsler-Bellevue IQ scores	IQ test scores (■)	Δ8 = -.17
		(varying in amounts of			from Rorschach psychograms		Δ9 = -.14
		clinical experience)
5b	Einhorn [27]	29 clinicians	I: 77 MMPI profiles	11	Judge the degree of neuroticism-	Actual diagnosis (■)	Δ10 = .02
	First study (This publication		II: 181 MMPI profiles		psychoticism		Δ110 = -.05
	Contains two studies)						(*, +, s)
7)	Todd (1955, see [29]), Note 3	10 clinical judges	78	19	Estimate patient IQ from the Rorschach	IQ test scores (■)	Δ12 = .05
					test
8)	Speroff, Connors, & Dawson	123 physicians:	440	32	Judge intensive care unit patients’	Patients’ actual	Δ13 = .05
	[30]	105 house staff,			hemodynamic status	hemodynamic status	(s)
		15 fellows,			(physicians’ estimation)
		3 attending physicians
	Novices:
6b)	Grebstein [28]	5 students	30	10	Judge Wechsler-Bellevue IQ scores	IQ test scores (■)	Δ14 = -.19
					from Rorschach psychograms based on
					paper profiles
b)	Business science, experts:
9)	Ashton [31]	13 executives, managers,	42	5	Predict advertising sales for Time	Actual advertising pages	Δ15 = .07
		sales personnel			magazine based on case descriptions	sold	(*, +, s)
10)	Roose & Doherty [32]	16 agency managers	200 / 160	64 / 5	Predict the success of life insurance	One-year criterion for	Δ16 = -.08
					salesmen based on paper profiles	success	(*, +, s)
11)	Goldberg [33]	43 bank loan officers	60	5	Predict bankruptcy experience based on	Actual bankruptcy	Δ17 = .03
					large corporation profiles	experience
12)	Kim, Chung, & Paradice [34]	3 experienced loan	119	7	Judge whether a firm would be able to	Actual financial data	I: Δ18 = .09
		officers	I: 60 big firms,		repay the loan requested based on		II: Δ19 = .02
			II: 59 small firms		financial profiles		(*, +, s)
13)	Mear & Firth [35]	38 professional security	30	10	Predict security returns based on	Actual security returns	Δ20 = .03
		analysts			financial profiles		(s)
14)	Ebert & Kruse [36]	5 securities analysts	35	22	Estimate future returns of common	Actual returns	Δ21 = .06
					stocks
15)	Wright [37]	47 students	50	4	Predict price changes for stocks from	Actual stock prices	Δ22 = .06
					1970 until 1971 based on paper profiles		(*, +, s)
					of securities
16)	Harvey & Harries [38]	24 psychology students	40	Not	Forecast sales outcomes based on paper	Actual sales outcome	Δ23 = -.07
	(1. experiment)			known	profiles		(s)
17)	Singh, 1990 [39]	52 business students	35	Not	Estimate of the stock price of a	Actual stock prices	Δ24 = .02
				known	company based on paper profiles		(s)
c)	Educational science, experts:
18)	Dawes [40]	1 admission committee	111	4	Admission decision for graduate school	Faculty ratings of l	Δ25 = .06
					based on paper profiles	performance in graduate
						school (▲)
19)	Cooksey, Freebody, & Davidson	20 teachers	118	5	Judge I: Reading comprehension	I-II: End-of-year test	I: Δ26 = .04
	[41]				And II: Word knowledge of	scores (■)	II: Δ27 = .04
					kindergarten children based on paper		(*, +, s)
					profiles
	Novices:
20)	Wiggins & Kohen [42]	98 psychology graduate	110	10	Forecast first-year-graduate grade point	Actual first-year-	Δ28 = .17
		students			averages based on paper profiles	graduate grade point	(s)
						averages
21)	Wiggins, Gregory, & Diller,	41 psychology students	90	10	Forecast first-year-graduate grade point	Actual first-year-	Δ29 = .06
	see Dawes and Corrigan [43],				averages based on paper profiles	graduate grade point
	repl. Wiggins and Kohen [42]					averages
22)	Athanasou & Cooksey [44]	18 technical and further	120	20	Judge whether students are interested in	Actual level of students’	Δ30 = .07
		education students			learning based on paper profile	interest	(*, +, s)
d)	Psychological science, experts:
23)	Szucko & Kleinmuntz [45]	6 experienced polygraph	30	3–4	Judge truthful / untruthful response	Actual theft	Δ31 = -.06
		interpreters			based on polygraph protocols		(*, +, s)
24)	Cooper & Werner [46]	18	33	17	Forecast violent behavior during the	Actual violent behavior	Δ32 = .00
		(9 psychologists,			first six months of incarceration based	during the first six	(s)
		9 case managers)			on inmates’ data forms	months of imprisonment
25)	Werner, Rose, Murdach, &	5 social workers	40	19	Predict imminent violence of	Actual violent acts	Δ33 = .03
	Yesavage [47]				psychiatric inpatients in the first 7 days	in the first 7 days	(*, +, s)
					following admission based on	following admission
					admission data
26)	Werner, Rose, & Yesavage [48]	30	40	19	Predict male patients’ violent behavior	Actual violence during	Δ34 = .06
		(15 psychologists,			during the first 7 days following	the first 7 days following	(s)
		15 psychiatrists)			admission based on case material	admission
	Novices:
27)	Gorman, Clover, & Doherty [49]	8 students	75:	I, III: 12	Predict students’ scores on an attitude	Actual data:	I: Δ35 = .73
			I, III: 50	II, IV: 6	scale (I, II) and a psychology	I, II: Attitude scale	II: Δ36 = .67
			II, IV: 25		examination (III, IV) based on	III, IV: Examination scale	III: Δ37 = .01
					interviews (I, III) and paper profiles	(■)	IV: Δ38 = .29
					(II, IV)		(*, s) (.08), see
							Camerer [6]
28)	Lehman [50]	14 students	40	19	Assess imminent violence of male	Actual violent acts in the	Δ39 = -.01
					patients in the first 7 days following	first 7 days following	(*, +, s)
					admission based on case material	admission

▲ = subjective criterion;

■ = test criterion;

(*) = idiographic approach (cumulating across individuals);

(*, +) = both research approaches are considered;

Δ = the success of bootstrapping models (see Eq 2); s = sub-sample of tasks for the second evaluation (psychometric corrected bootstrapping models).