Table 2:
The simulated prediction (test set) median accuracy and interquartile range of models for N=100, 250 and 500 patients are displayed within Table 2 for small and large random effects, for linear and tree data generating processes, and for 2, 4 and 7 repeated measurements per patient. Traditional CART, Bayesian GLMM, BiMM Tree with one iteration (denoted BiMM Tree 1) and BiMM Tree with multiple iterations for the split function maximizing sensitivity and the general split function (denoted BiMM Tree H1 and BiMM Tree H3) are compared.
| Model | Repeated Outcomes |
N=100 | N=250 | N=500 | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Linear DGP | Tree DGP | Linear DGP | Tree DGP | Linear DGP | Tree DGP | |||||||||||
| Small RE | Large RE | Small RE | Large RE | Small RE | Large RE | Small RE | Large RE | Small RE | Large RE | Small RE | Large RE | |||||
| CART | 2 | 0.913 (0.890,0.925) |
0.678 (0.651.0.699) |
0.963 (0.950,0.971) |
0.769 (0.745,0.791) |
0.929 (0.920,0.937) |
0.706 (0.690,0.722) |
0.970 (0.965,0.975) |
0.796 (0.780,0.812) |
0.942 (0.936,0.948) |
0.732 (0.718,0.745) |
0.971 (0.965,0.975) |
0.821 (0.806,0.832) |
|||
| 4 | 0.933 (0.922,0.940) |
0.679 (0.659,0.699) |
0.973 (0.966,0.978) |
0.805 (0.788,0.821) |
0.942 (0.937,0.947) |
0.721 (0.707,0.733) |
0.975 (0.970,0.979) |
0.830 (0.818,0.843) |
0.945 (0.941,0.950) |
0.735 (0.724,0.745) |
0.975 (0.971,0.980) |
0.844 (0.833,0.853) |
||||
| 7 | 0.941 (0.935,0.947) |
0.697 (0.680,0.714) |
0.977 (0.972,0.981) |
0.828 (0.814,0.842) |
0.946 (0.942,0.950) |
0.732 (0.720,0.743) |
0.979 (0.975,0.982) |
0.853 (0.840,0.863) |
0.947 (0.944,0.951) |
0.740 (0.729,0.750) |
0.979 (0.975,0.982) |
0.859 (0.849,0.867) |
||||
| Bayesian GLMM |
2 | 0.924 (0.914,0.933) |
0.715 (0.700,0.730) |
0.820 (0.805,0.835) |
0.709 (0.691,0.723) |
0.939 (0.933,0.944) |
0.731 (0.718,0.745) |
0.836 (0.826,0.847) |
0.722 (0.708,0.733) |
0.943 (0.938,0.948) |
0.737 (0.723,0.748) |
0.842 (0.833,0.851) |
0.725 (0.714,0.737) |
|||
| 4 | 0.922 (0.915,0.928) |
0.704 (0.689,0.716) |
0.805 (0.795,0.814) |
0.728 (0.717,0.738) |
0.930, (0.926,0.934) |
0.717 (0.704,0.729) |
0.812 (0.804,0.820) |
0.735 (0.724,0.744) |
0.932 (0.928,0.936) |
0.721 (0.710,0.732) |
0.814 (0.808,0.821) |
0.736 (0.727,0.746) |
||||
| 7 | 0.917 (0.912,0.921) |
0.695 (0.682,0.707) |
0.805 (0.796,0.812) |
0.750 (0.741,0.760) |
0.921 (0.917,0.925) |
0.707 (0.696,0.717) |
0.808 (0.801,0.814) |
0.755 (0.746,0.763) |
0.923 (0.919,0.926) |
0.710 (0.699,0.721) |
0.809 (0.803,0.814) |
0.756 (0.748,0.765) |
||||
| BiMM Tree 1 Iteration |
2 | 0.849 (0.833,0.868) |
0.827 (0.776,0.852) |
0.916 (0.897,0.927) |
0.836 (0.782,0.902) |
0.850 (0.838,0.866) |
0.850 (0.830,0.873) |
0.921 (0.914,0.929) |
0.911 (0.880,0.923) |
0.850 (0.840,0.863) |
0.856 (0.840,0.886) |
0.920 (0.915,0.926) |
0.917 (0.907,0.925) |
|||
| 4 | 0.845 (0.822,0.860) |
0.815 (0.780,0.849) |
0.917 (0.881,0.952) |
0.901 (0.854,0.956) |
0.850 (0.835,0.862) |
0.837 (0.807,0.862) |
0.942 (0.888,0.959) |
0.947 (0.894,0.964) |
0.849 (0.838,0.862) |
0.841 (0.813,0.865) |
0.952 (0.891,0.960) |
0.953 (0.900,0.963) |
||||
| 7 | 0.881 (0.869,0.891) |
0.837 (0.806,0.864) |
0.902 (0.892,0.913) |
0.905 (0.862,0.920) |
0.887 (0.878,0.894) |
0.860 (0.837,0.889) |
0.905 (0.895,0.914) |
0.907 (0.864,0.914) |
0.890 (0.884,0.895) |
0.867 (0.843,0.890) |
0.905 (0.897,0.913) |
0.906 (0.858,0.913) |
||||
| BiMM Tree H1 Algorithm |
2 | 0.852 (0.832,0.876) |
0.840 (0.813,0.854) |
0.910 (0.842,0.924) |
0.809 (0.757,0.870) |
0.856 (0.842,0.874) |
0.847 (0.836,0.861) |
0.918 (0.904,0.925) |
0.858 (0.819,0.916) |
0.859 (0.845,0.874) |
0.851 (0.841,0.866) |
0.918 (0.909,0.924) |
0.852 (0.822,0.917) |
|||
| 4 | 0.844 (0.823,0.857) |
0.806 (0.790,0.840) |
0.882 (0.868,0.925) |
0.871 (0.817,0.934) |
0.847 (0.830,0.860) |
0.820 (0.800,0.850) |
0.887 (0.873,0.947) |
0.882 (0.855,0.952) |
0.847 (0.832,0.860) |
0.816 (0.796,0.852) |
0.886 (0.874,0.901) |
0.878 (0.851,0.904) |
||||
| 7 | 0.879 (0.862,0.891) |
0.835 (0.814,0.847) |
0.899 (0.890,0.911) |
0.891 (0.801,0.910) |
0.887 (0.876,0.894) |
0.842 (0.832,0.852) |
0.905 (0.894,0.913) |
0.897 (0.853,0.909) |
0.890 (0.884,0.895) |
0.843 (0.836,0.851) |
0.905 (0.896,0.912) |
0.896 (0.725,0.907) |
||||
| BiMM Tree H3 Algorithm |
2 | 0.849 (0.834,0.867) |
0.838 (0.804,0.852) |
0.913 (0.850,0.924) |
0.834 (0.775,0.909) |
0.850 (0.839,0.865) |
0.846 (0.835,0.857) |
0.920 (0.912,0.928) |
0.908 (0.830,0.923) |
0.850 (0.840,0.863) |
0.848 (0.838,0.859) |
0.920 (0.914,0.926) |
0.915 (0.852,0.923) |
|||
| 4 | 0.843 (0.822,0.857) |
0.803 (0.788,0.833) |
0.881 (0.868,0.938) |
0.874 (0.848,0.943) |
0.848 (0.832,0.861) |
0.807 (0.793,0.842) |
0.881 (0.871,0.949) |
0.879 (0.860,0.953) |
0.848 (0.836,0.861) |
0.805 (0.793,0.839) |
0.879 (0.869,0.891) |
0.877 (0.862,0.894) |
||||
| 7 | 0.879 (0.852,0.891) |
0.833 (0.805,0.845) |
0.895 (0.887,0.905) |
0.891 (0.805,0.909) |
0.886 (0.877,0.894) |
0.840 (0.831,0.849) |
0.899 (0.890,0.909) |
0.895 (0.785,0.907) |
0.890 (0.884,0.895) |
0.842 (0.835,0.849) |
0.890 (0.891,0.910) |
0.895 (0.776,0.904) |
||||