Table 5.

Mean Rate Ratios, Empirical Variance, Mean Squared Error, and Percent Coverage of 95% Confidence Intervals From 5,000 Simulated Studies With Unmeasured Confounding Leading to Treatment Withheld According to AUC of the Propensity Score Model^a

	AUC of the Propensity Score Model
	AUC = 0.65				AUC = 0.75 b				AUC = 0.85
Weighting and Trimming Method	RR	Var c	MSE d	Cov, % e	RR	Var c	MSE d	Cov, % e	RR	Var c	MSE d	Cov, % e
Crude	2.20	0.0053	0.0147	52	2.87	0.0077	0.1386	0	4.12	0.0068	0.5290	0
Residual confounding	1.52	0.0048	0.0796	92	1.31	0.0067	0.1829	0	1.37	0.0066	0.1472	0
True outcome model	2.00	0.0028	0.0028	96	2.00	0.0034	0.0034	95	2.00	0.0029	0.0029	96
IPTW
No trimming	1.57	0.0046	0.0620	7	1.31	0.0082	0.1867	0	1.19	0.0227	0.2883	2
Common rangef	1.57	0.0045	0.0625	7	1.30	0.0080	0.1941	0	1.15	0.0215	0.3259	1
Stürmerf	1.89	0.0043	0.0078	89	1.96	0.0060	0.0063	96	1.99	0.0069	0.0069	96
Walkerf	1.80	0.0043	0.0151	70	2.00	0.0068	0.0068	96	1.99	0.0095	0.0095	96
Crumpf	1.57	0.0046	0.0630	7	1.28	0.0080	0.2040	0	1.23	0.0119	0.2483	0
SMR-treated weights
No trimming	1.38	0.0074	0.1438	1	0.99	0.0172	0.5157	0	0.79	0.0444	0.9151	0
Common range	1.38	0.0072	0.1444	1	0.98	0.0162	0.5264	0	0.76	0.0411	0.9686	0
Stürmer	1.84	0.0049	0.0118	81	1.94	0.0065	0.0074	94	2.00	0.0067	0.0067	96
Walker	1.72	0.0053	0.0278	48	2.00	0.0068	0.0068	96	2.00	0.0094	0.0094	96
Crump	1.39	0.0073	0.1417	2	1.01	0.0164	0.4876	0	0.93	0.0220	0.6025	0
SMR-untreated weights
No trimming	1.65	0.0041	0.0415	19	1.54	0.0061	0.0728	12	1.76	0.0119	0.0289	78
Common range	1.65	0.0041	0.0419	19	1.53	0.0060	0.0772	10	1.70	0.0111	0.0372	68
Stürmer	1.90	0.0043	0.0070	91	1.97	0.0063	0.0065	96	1.99	0.0078	0.0078	96
Walker	1.83	0.0042	0.0123	77	2.00	0.0071	0.0072	96	1.99	0.0101	0.0101	96
Crump	1.64	0.0041	0.0427	17	1.50	0.0055	0.0899	4	1.63	0.0063	0.0475	32
Overlap weights
No trimming	1.47	0.0056	0.1013	2	1.20	0.0083	0.2661	0	1.25	0.0092	0.2293	0
Common range	1.47	0.0056	0.1014	2	1.20	0.0082	0.2673	0	1.25	0.0092	0.2297	0
Stürmer	1.86	0.0046	0.0102	84	1.95	0.0060	0.0066	95	2.00	0.0062	0.0062	96
Walker	1.75	0.0049	0.0225	55	2.00	0.0066	0.0066	96	1.99	0.0090	0.0090	96
Crump	1.47	0.0056	0.1005	2	1.21	0.0083	0.2621	0	1.30	0.0096	0.1957	1
Matching weights
No trimming	1.39	0.0070	0.1397	1	1.15	0.0089	0.3141	0	1.31	0.0084	0.1882	0
Common range	1.39	0.0069	0.1399	1	1.15	0.0088	0.3154	0	1.31	0.0084	0.1885	0
Stürmer	1.84	0.0049	0.0118	81	1.94	0.0065	0.0074	94	2.00	0.0067	0.0067	96
Walker	1.72	0.0053	0.0278	48	2.00	0.0068	0.0068	96	2.00	0.0094	0.0094	96
Crump	1.39	0.0070	0.1379	1	1.16	0.0090	0.3102	0	1.35	0.0088	0.1614	1
Entropy weights
No trimming	1.23	0.0101	0.2470	0	1.23	0.0081	0.2472	0	1.50	0.0053	0.0898	3
Common range	1.23	0.0100	0.2489	0	1.22	0.0080	0.2489	0	1.49	0.0052	0.0899	3
Stürmer	2.00	0.0062	0.0062	96	1.95	0.0059	0.0064	95	1.86	0.0045	0.0094	86
Walker	1.99	0.0090	0.0090	96	2.00	0.0065	0.0065	96	1.76	0.0047	0.0203	59
Crump	1.28	0.0100	0.2088	0	1.23	0.0082	0.2468	0	1.50	0.0053	0.0894	3

Abbreviations: AUC, area under the receiver operating characteristic curve; Cov, coverage; IPTW, inverse probability of treatment weighting; MSE, mean squared error; RR, rate ratio; SMR, standardized mortality ratio; Var, variance.

^a True RR = 2.0; treatment prevalence = 20%.

^b See Table 4.

^c Variance of treatment effect estimates (log(RR)) over 5,000 simulated studies.

^d Mean of (log(RR) − log(2.0))² over 5,000 simulated studies.

^e Percentage of simulated studies in which the 95% confidence interval includes the true value (RR = 2.0).

^f Common range: restricting to positivity; Stürmer: asymmetrical trimming at fifth percentile of treated and 95th percentile of untreated; Walker: preference score trimming below 0.3 and above 0.7; Crump: propensity score trimming below 0.1 and above 0.9.