Table 4.
Differences Between Fitted and Actual Values of EC50, Slope, ymin, and ymax by XLfit without Prefit
| Parameter | No1 | No2 | No3 | No4 | Median Dif | Max Dif | pvalue | r |
|---|---|---|---|---|---|---|---|---|
| EC50 | 10 | 0 | 0 | 8 | 0 | 3.58 | <2.2e-16 | 0.775 |
| 10 | 1 | 0 | 22 | 0.13 | 37.17 | 0.2758 | 0.11 | |
| 10 | 2 | 0 | 39 | 1.41 | 10.58 | 0.02382 | 0.226 | |
| 10 | 3 | 0 | 49 | 1.93 | 10.4 | 0.01612 | 0.24 | |
| 10 | 4 | 0 | 40 | 1.56 | 24.79 | 0.009522 | 0.258 | |
| 20 | 0 | 0 | 6 | 0 | 3.61 | <2.2e-16 | 0.929 | |
| 20 | 2 | 0 | 14 | 0.09 | 11.73 | <2.2e-16 | 0.709 | |
| 20 | 4 | 0 | 11 | 0.19 | 8.3 | 1.23E-12 | 0.635 | |
| 20 | 6 | 0 | 13 | 0.22 | 10.42 | <2.2e-16 | 0.73 | |
| 20 | 8 | 0 | 19 | 0.34 | 12.78 | 4.53E-08 | 0.514 | |
| slope | 10 | 0 | 0 | 32 | 0 | 4.82 | 8.28E-07 | 0.47 |
| 10 | 1 | 0 | 46 | 1.13 | 6.84E+15 | 0.0933 | -0.169 | |
| 10 | 2 | 0 | 75 | 2.3 | 189.41 | 0.9782 | 0.003 | |
| 10 | 3 | 0 | 78 | 2.02 | 120.51 | 0.4397 | 0.078 | |
| 10 | 4 | 0 | 76 | 3.21 | 253.95 | 0.878 | 0.016 | |
| 20 | 0 | 0 | 15 | 0 | 4.48 | <2.2e-16 | 0.746 | |
| 20 | 2 | 0 | 41 | 0.89 | 66.5 | 0.9688 | 0.004 | |
| 20 | 4 | 0 | 55 | 1.34 | 56.63 | 0.4695 | 0.073 | |
| 20 | 6 | 0 | 57 | 1.56 | 135.52 | 0.02047 | 0.232 | |
| 20 | 8 | 0 | 67 | 1.9 | 104.98 | 0.4246 | 0.081 | |
| ymin | 10 | 0 | 0 | 3 | 0 | 1.79E+03 | 0.06308 | 0.187 |
| 10 | 1 | 0 | 13 | 5.73 | 2.75E+10 | 0.3146 | 0.102 | |
| 10 | 2 | 0 | 5 | 13.25 | 2.77E+03 | 0.4586 | 0.075 | |
| 10 | 3 | 0 | 5 | 13.46 | 4.79E+04 | 0.8188 | 0.023 | |
| 10 | 4 | 0 | 9 | 14.21 | 1.99E+03 | 0.7425 | 0.033 | |
| 20 | 0 | 0 | 4 | 0 | 976 | 0.534 | 0.063 | |
| 20 | 2 | 0 | 3 | 2.96 | 762.23 | 0.3416 | 0.096 | |
| 20 | 4 | 0 | 1 | 3.85 | 358.8 | 0.01858 | 0.235 | |
| 20 | 6 | 0 | 4 | 4.79 | 7.93E+03 | 0.325 | 0.099 | |
| 20 | 8 | 0 | 5 | 6.42 | 6.71E+03 | 0.6725 | 0.043 | |
| ymax | 10 | 0 | 0 | 3 | 0 | 2.23E+03 | 0.1889 | -0.132 |
| 10 | 1 | 0 | 1 | 5.49 | 1.86E+03 | 0.01008 | 0.256 | |
| 10 | 2 | 0 | 19 | 48.94 | 1.01E+06 | 0.1667 | -0.139 | |
| 10 | 3 | 0 | 30 | 75.75 | 7.36E+07 | 0.895 | 0.013 | |
| 10 | 4 | 0 | 30 | 75.74 | 1.69E+08 | 0.5535 | 0.06 | |
| 20 | 0 | 0 | 0 | 0 | 99.51 | <2.2e-16 | 0.81 | |
| 20 | 2 | 0 | 7 | 3.82 | 3.12E+04 | 0.6598 | -0.045 | |
| 20 | 4 | 0 | 9 | 6.57 | 4.59E+07 | 0.6877 | -0.041 | |
| 20 | 6 | 0 | 8 | 6.12 | 1.66E+07 | 0.4453 | 0.077 | |
| 20 | 8 | 0 | 5 | 7.02 | 2.77E+04 | 0.3871 | -0.087 |
No1: number of data points; No2: number of randomly perturbed data points; No3: number of curves without fit; No4: number of curves with absolute differences in EC50, ymin and ymax > 2.0, 200.0% and 200.0%, respectively and maximum difference in ratio of slope > 2.0 or < 0.5; Median Dif: median difference; Max Dif: maximum difference; pvalue: significance of Pearson correlations; and r: Pearson correlation coefficient.