Table 3.
Differences Between Fitted and Actual Values of EC50, Slope, ymin and ymax by XLfit with Prefit
| Parameter | No1 | No2 | No3 | No4 | Median Dif | Max Dif | pvalue | r |
|---|---|---|---|---|---|---|---|---|
| EC50 | 10 | 0 | 0 | 0 | 0 | 5.00E-04 | <2.2e-16 | 1.000 |
| 10 | 1 | 0 | 22 | 0.13 | 37.17 | 0.2758 | 0.110 | |
| 10 | 2 | 0 | 38 | 1.27 | 60.67 | 0.2034 | -0.128 | |
| 10 | 3 | 0 | 38 | 1.28 | 58.79 | 0.3286 | 0.099 | |
| 10 | 4 | 0 | 45 | 1.69 | 45.08 | 0.3929 | 0.086 | |
| 20 | 0 | 0 | 0 | 0 | 1.41 | <2.2e-16 | 0.996 | |
| 20 | 2 | 1 | 11 | 0.05 | 226.3 | 0.1308 | -0.153 | |
| 20 | 4 | 0 | 14 | 0.13 | 66.41 | 0.0007174 | 0.333 | |
| 20 | 6 | 0 | 18 | 0.19 | 26.76 | 0.003044 | 0.293 | |
| 20 | 8 | 0 | 17 | 0.25 | 53.92 | 0.7481 | -0.033 | |
| slope | 10 | 0 | 0 | 0 | 0 | 4.96E-04 | <2.2e-16 | 1.000 |
| 10 | 1 | 0 | 46 | 1.13 | 6.84E+15 | 0.0933 | -0.169 | |
| 10 | 2 | 0 | 69 | 3.54 | 4.47E+29 | 0.7961 | -0.026 | |
| 10 | 3 | 0 | 75 | 4.36 | 1.83E+66 | 0.3176 | 0.101 | |
| 10 | 4 | 0 | 78 | 8.79 | 7.84E+14 | 0.2788 | -0.109 | |
| 20 | 0 | 0 | 0 | 0 | 1.16 | <2.2e-16 | 0.994 | |
| 20 | 2 | 1 | 27 | 0.63 | 2.66E+22 | 0.5388 | 0.063 | |
| 20 | 4 | 0 | 43 | 1.21 | 1.85E+35 | 0.3934 | -0.086 | |
| 20 | 6 | 0 | 55 | 1.75 | 1.08E+17 | 0.1767 | 0.136 | |
| 20 | 8 | 0 | 61 | 2.05 | 3.52E+21 | 0.1286 | -0.153 | |
| ymin | 10 | 0 | 0 | 0 | 0 | 4.96E-04 | <2.2e-16 | 1.000 |
| 10 | 1 | 0 | 13 | 5.73 | 2.75E+10 | 0.3146 | 0.102 | |
| 10 | 2 | 0 | 13 | 12.1 | 3.05E+04 | 0.759 | -0.031 | |
| 10 | 3 | 0 | 13 | 10.24 | 3.09E+07 | 0.1349 | -0.151 | |
| 10 | 4 | 0 | 14 | 14.36 | 1.74E+10 | 0.6675 | 0.043 | |
| 20 | 0 | 0 | 1 | 0 | 9.97E+04 | 0.3379 | -0.097 | |
| 20 | 2 | 1 | 4 | 2.21 | 1.21E+09 | 0.5864 | -0.055 | |
| 20 | 4 | 0 | 4 | 3.45 | 1.06E+08 | 0.5062 | 0.067 | |
| 20 | 6 | 0 | 5 | 4.7 | 2.00E+03 | 0.7532 | -0.032 | |
| 20 | 8 | 0 | 5 | 5.49 | 2.76E+03 | 0.9608 | 0.005 | |
| ymax | 10 | 0 | 0 | 0 | 0 | 4.97E-04 | <2.2e-16 | 1.000 |
| 10 | 1 | 0 | 1 | 5.49 | 1.86E+03 | 0.01008 | 0.256 | |
| 10 | 2 | 0 | 7 | 11.21 | 2.04E+04 | 0.1812 | -0.135 | |
| 10 | 3 | 0 | 12 | 17.65 | 6.06E+05 | 0.5637 | -0.058 | |
| 10 | 4 | 0 | 12 | 25.25 | 1.89E+04 | 0.07925 | -0.176 | |
| 20 | 0 | 0 | 0 | 0 | 0.09 | <2.2e-16 | 1.000 | |
| 20 | 2 | 1 | 4 | 2.19 | 2.61E+03 | 0.872 | 0.016 | |
| 20 | 4 | 0 | 6 | 4.44 | 2.01E+04 | 0.6786 | 0.042 | |
| 20 | 6 | 0 | 5 | 5.85 | 9.80E+04 | 0.7925 | 0.027 | |
| 20 | 8 | 0 | 6 | 5.9 | 4.93E+07 | 0.2669 | -0.112 |
No1: number of data points; No2: number of randomly perturbed data points; No3: number of curves without fit; No4: number of curves with absolute differences in EC50, ymin and ymax > 2.0, 200.0% and 200.0%, respectively and maximum difference in ratio of slope > 2.0 or < 0.5; Median Dif: median difference; Max Dif: maximum difference; pvalue: significance of Pearson correlations; and r: Pearson correlation coefficient.