Skip to main content
. 2023 Jul 19;23(14):6509. doi: 10.3390/s23146509

Table 6.

Tool performance comparison for the tasks of machine learning and statistical query. For each criterion, we provide the ranking (in the upper part of the table cells) of different tools associated with the numerical performance (in the lower part of the table cells) under the considered criterion.

Performance Criteria (RMPSE)
Categorical Data Continuous Data
Utility FES Runtime FES Memory FES Utility RES Runtime RES Memory RES Utility DFC Runtime DFC Memory DFC Utility FES Runtime FES Memory FES Utility RES Runtime RES Memory RES Utility DFC Runtime DFC Memory DFC
machine learning Diffprivlib 1st
0.15		 3rd
797.86		 3rd
6.35		 3rd
6.4 ×108 		3rd
1176.20		 3rd
46.21		 3rd
2.76
to
1.21 ×108 		3rd
603.17
to
1.15 ×103 		3rd
9.11
to
16.64		 Results far away from usable
4.6 ×108 2323.46 35.64 1.96 ×1011 856.53 31.54 1.53 ×109
to
1.89 ×1018 		603.17
to
1.15 ×103 		40.88
to
45.38
PyTorch
Opacus		 3rd
4.47		 2nd
221.13		 2nd
0.72		 1st
1283.47		 2nd
217.85		 1st
2.35		 1st
3.44
to
7.10		 2nd
197.67
to
211.96		 1st
0.11
to
0.16		 2nd
2.59		 2nd
213.23		 1st
1.77		 1st
5122.05		 2nd
224.95		 1st
1.81		 2nd
3.99
to
38.21		 2nd
199.77
to
213.90		 1st
0.55
to
1.68
TensorFlow
Privacy		 2nd
2.91		 1st
22.52		 1st
6.1  ×10−4 		2nd
6960.15		 1st
17.25		 2rd
9.74		 2rd
4.65
to
12.73		 1st
10.30
to
14.23		 2nd
0.64
to
7.06		 1st
0.90		 1st
35.69		 2nd
15.77		 2nd
6177.35		 1st
1.57		 2nd
28.32		 1st
2.32
to
26.37		 1st
21.01
to
35.25		 2nd
13.48
to
26.88
statistical query Google
Differential
Privacy		 1st
0.61		 1st
115.63		 1st
8.12		 1st
74.3		 1st
137.37		 1st
25.19		 1st
0.016
to
0.31		 1st
108.25
to
129.35		 1st
0.05
to
1.88		 1st
0.5		 1st
120.32		 1st
4.6  ×10−4 		1st
75.9		 1st
126.63		 1st
7.6 ×10−5 		1st
0.02
to
0.36		 1st
115.11
to
137.43		 1st
0
to
2.16
OpenDP
SmartNoise		 2rd
2.9		 2rd
481.11		 2rd
14.08		 2nd
801		 2rd
413.96		 2rd
29.27		 2rd
0.11
to
2.08		 2rd
408.74
to
466.23		 2rd
0.50
to
2.75		 2rd
3.9		 2rd
473.63		 2rd
4.56		 2nd
746		 2rd
558.16		 2rd
4.67		 2rd
0.19
to
2.24		 2rd
427.19
to
465.05		 2rd
5.06
to
6.47