. Author manuscript; available in PMC: 2017 Aug 10.

Published in final edited form as: Proc CGO. 2010 Apr;2010:230–237. doi: 10.1145/1772954.1772987

Table 2.

Summary of Gappa++ (above) and simulated (below) error bounds for IIR filter. Uniformly distributed [0,1] input rounded to model data from 12-bit ADC.

DF I	Actual	Ideal Coeffs.	Ideal Arith.
32-bit float	±9.77e-4	±9.71e-4	±1.10e-5
32-bit float	±2.02e-5
16-bit fix	±5.77e-2	±5.51e-2	±2.76e-3
16-bit fix	±3.15e-3

DF II	Actual	Ideal Coeffs.	Ideal Arith.

32-bit float	±9.70e-4	±9.64e-4	±1.10e-5
32-bit float	±6.15e-5
16-bit fix	±2.94e-3	±2.61e-4	±2.76e-3
16-bit fix	±6.42e-4