Skip to main content
. 2010 Oct 21;4:57–66. doi: 10.2174/1875397301004010057

Table 4.

Differences Between Fitted and Actual Values of EC50, Slope, ymin, and ymax by XLfit without Prefit

Parameter No1 No2 No3 No4 Median Dif Max Dif pvalue r
EC50 10 0 0 8 0 3.58 <2.2e-16 0.775
10 1 0 22 0.13 37.17 0.2758 0.11
10 2 0 39 1.41 10.58 0.02382 0.226
10 3 0 49 1.93 10.4 0.01612 0.24
10 4 0 40 1.56 24.79 0.009522 0.258
20 0 0 6 0 3.61 <2.2e-16 0.929
20 2 0 14 0.09 11.73 <2.2e-16 0.709
20 4 0 11 0.19 8.3 1.23E-12 0.635
20 6 0 13 0.22 10.42 <2.2e-16 0.73
20 8 0 19 0.34 12.78 4.53E-08 0.514
slope 10 0 0 32 0 4.82 8.28E-07 0.47
10 1 0 46 1.13 6.84E+15 0.0933 -0.169
10 2 0 75 2.3 189.41 0.9782 0.003
10 3 0 78 2.02 120.51 0.4397 0.078
10 4 0 76 3.21 253.95 0.878 0.016
20 0 0 15 0 4.48 <2.2e-16 0.746
20 2 0 41 0.89 66.5 0.9688 0.004
20 4 0 55 1.34 56.63 0.4695 0.073
20 6 0 57 1.56 135.52 0.02047 0.232
20 8 0 67 1.9 104.98 0.4246 0.081
ymin 10 0 0 3 0 1.79E+03 0.06308 0.187
10 1 0 13 5.73 2.75E+10 0.3146 0.102
10 2 0 5 13.25 2.77E+03 0.4586 0.075
10 3 0 5 13.46 4.79E+04 0.8188 0.023
10 4 0 9 14.21 1.99E+03 0.7425 0.033
20 0 0 4 0 976 0.534 0.063
20 2 0 3 2.96 762.23 0.3416 0.096
20 4 0 1 3.85 358.8 0.01858 0.235
20 6 0 4 4.79 7.93E+03 0.325 0.099
20 8 0 5 6.42 6.71E+03 0.6725 0.043
ymax 10 0 0 3 0 2.23E+03 0.1889 -0.132
10 1 0 1 5.49 1.86E+03 0.01008 0.256
10 2 0 19 48.94 1.01E+06 0.1667 -0.139
10 3 0 30 75.75 7.36E+07 0.895 0.013
10 4 0 30 75.74 1.69E+08 0.5535 0.06
20 0 0 0 0 99.51 <2.2e-16 0.81
20 2 0 7 3.82 3.12E+04 0.6598 -0.045
20 4 0 9 6.57 4.59E+07 0.6877 -0.041
20 6 0 8 6.12 1.66E+07 0.4453 0.077
20 8 0 5 7.02 2.77E+04 0.3871 -0.087

No1: number of data points; No2: number of randomly perturbed data points; No3: number of curves without fit; No4: number of curves with absolute differences in EC50, ymin and ymax > 2.0, 200.0% and 200.0%, respectively and maximum difference in ratio of slope > 2.0 or < 0.5; Median Dif: median difference; Max Dif: maximum difference; pvalue: significance of Pearson correlations; and r: Pearson correlation coefficient.