Table 5.
Performance metrics for the assessment of cross-platform transferability of signature genes of k-NN models based on the SEQC NB data
| Direction | Endpoint | Gene set | Microarray models Is predict microarray validation samples | RNA-Seq models predict RNA-Seq validation samples | T-index | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Accuracy | AUC | Accuracy | AUC | ||||||||
| Mean | 95% CI | Mean | 95% CI | Mean | 95% CI | Mean | 95% CI | ||||
| From microarray to RNA-Seq | A* | A | 0.732 | 0.667-0.775 | 0.708 | 0.645-0.758 | 0.728 | 0.653-0.771 | 0.696 | 0.627-0.746 | 0.729 |
| B | 0.721 | 0.655-0.759 | 0.691 | 0.631-0.733 | 0.714 | 0.647-0.759 | 0.678 | 0.614-0.726 | 0.716 | ||
| C | 0.696 | 0.639-0.735 | 0.664 | 0.609-0.704 | 0.702 | 0.654-0.747 | 0.670 | 0.631-0.713 | 0.700 | ||
| B* | A | 0.780 | 0.747-0.811 | 0.667 | 0.615-0.72 | 0.787 | 0.755-0.815 | 0.648 | 0.601-0.694 | 0.786 | |
| B | 0.777 | 0.743-0.807 | 0.643 | 0.585-0.704 | 0.794 | 0.755-0.827 | 0.659 | 0.580-0.725 | 0.791 | ||
| C | 0.789 | 0.739-0.827 | 0.678 | 0.613-0.732 | 0.788 | 0.747-0.819 | 0.660 | 0.620-0.711 | 0.788 | ||
| C* | A | 0.978 | 0.971-0.992 | 0.978 | 0.971-0.992 | 0.992 | 0.992-0.992 | 0.992 | 0.992-0.992 | 0.992 | |
| B | 0.989 | 0.983-0.992 | 0.989 | 0.984-0.992 | 0.989 | 0.983-0.992 | 0.989 | 0.984-0.992 | 0.989 | ||
| C | 0.931 | 0.900-0.954 | 0.935 | 0.905-0.958 | 0.988 | 0.975-0.992 | 0.988 | 0.976-0.992 | 0.984 | ||
| D* | A | 0.934 | 0.882-0.971 | 0.920 | 0.863-0.962 | 0.921 | 0.882-0956 | 0.895 | 0.848-0.940 | 0.921 | |
| B | 0.947 | 0.824-0.978 | 0.938 | 0.798-0.973 | 0.933 | 0.772-0.978 | 0.915 | 0.737-0.967 | 0.934 | ||
| C | 0.915 | 0.838-0.956 | 0.911 | 0.841-0.961 | 0.921 | 0.853-0.963 | 0.914 | 0.840-0.956 | 0.920 | ||
| E* | A | 0.624 | 0.533-0.700 | 0.534 | 0.463-0.615 | 0.606 | 0.522-0.689 | 0.537 | 0.460-0.617 | 0.612 | |
| B | 0.562 | 0.478-0.623 | 0.507 | 0.417-0.588 | 0.569 | 0.500-0.633 | 0.519 | 0.443-0.595 | 0.566 | ||
| C | 0.607 | 0.511-0.689 | 0.513 | 0.437-0.603 | 0.599 | 0.511-0.689 | 0.515 | 0.438-0.603 | 0.602 | ||
| F* | A | 0.513 | 0.444-0.589 | 0.513 | 0.446-0.585 | 0.510 | 0.433-0.589 | 0.510 | 0.426-0.589 | 0.511 | |
| B | 0.507 | 0.456-0.567 | 0.511 | 0.452-0.572 | 0.490 | 0.422-0.556 | 0.498 | 0.431-0.562 | 0.498 | ||
| C | 0.532 | 0.456-0.611 | 0.534 | 0.460-0.609 | 0.527 | 0.444-0.611 | 0.531 | 0.448-0.611 | 0.529 | ||
| From RNA-Seq to microarray | A* | A | 0.701 | 0.614-0.759 | 0.669 | 0.590-0.732 | 0.709 | 0.643-0.759 | 0.671 | 0.603-0.727 | 0.703 |
| B | 0.680 | 0.584-0.741 | 0.644 | 0.543-0.708 | 0.693 | 0.604-0.747 | 0.651 | 0.568-0.710 | 0.684 | ||
| C | 0.719 | 0.647-0.767 | 0.691 | 0.618-0.748 | 0.730 | 0.648-0.771 | 0.698 | 0.621-0.743 | 0.722 | ||
| B* | A | 0.775 | 0.715-0.819 | 0.662 | 0.576-0.737 | 0.775 | 0.733-0.811 | 0.640 | 0.579-0.696 | 0.775 | |
| B | 0.777 | 0.715-0.819 | 0.639 | 0.552-0.713 | 0.785 | 0.735-0.825 | 0.640 | 0.566-0.718 | 0.778 | ||
| C | 0.792 | 0.749-0.823 | 0.681 | 0.621-0.757 | 0.790 | 0.753-0.819 | 0.659 | 0.606-0.728 | 0.791 | ||
| C* | A | 0.971 | 0.967-0.992 | 0.972 | 0.969-0.992 | 0.984 | 0.983-0.992 | 0.985 | 0.984-0.992 | 0.971 | |
| B | 0.939 | 0.900-0.950 | 0.943 | 0.901-0.954 | 0.990 | 0.988-0.992 | 0.990 | 0.988-0.992 | 0.939 | ||
| C | 0.987 | 0.975-0.992 | 0.987 | 0.975-0.992 | 0.987 | 0.971-0.992 | 0.987 | 0.972-0.992 | 0.987 | ||
| D* | A | 0.914 | 0.809-0.971 | 0.899 | 0.792-0.957 | 0.927 | 0.868-0.963 | 0.910 | 0.841-0.951 | 0.915 | |
| B | 0.918 | 0.838-0.963 | 0.907 | 0.825-0.962 | 0.928 | 0.860-0.971 | 0.908 | 0.836-0.957 | 0.918 | ||
| C | 0.937 | 0.875-0.978 | 0.923 | 0.853-0.973 | 0.933 | 0.882-0.971 | 0.913 | 0.882-0.962 | 0.937 | ||
| E* | A | 0.598 | 0.456-0.689 | 0.506 | 0.414-0.606 | 0.598 | 0.444-0.700 | 0.522 | 0.400-0.630 | 0.598 | |
| B | 0612 | 0.511-0.700 | 0.513 | 0.432-0.602 | 0.588 | 0.489-0.678 | 0.510 | 0.415-0.613 | 0.603 | ||
| C | 0.582 | 0.478-0.667 | 0.512 | 0.412-0.633 | 0.596 | 0.500-0.689 | 0.522 | 0.432-0.615 | 0.587 | ||
| F* | A | 0.507 | 0.433-0.600 | 0.502 | 0.421-0.590 | 0.498 | 0.400-0.600 | 0.496 | 0.406-0.597 | 0.503 | |
| B | 0.500 | 0.411-0.595 | 0.503 | 0.414-0.596 | 0.493 | 0.400-0.589 | 0.503 | 0.411-0.601 | 0.496 | ||
| C | 0.523 | 0.456-0.600 | 0.527 | 0.452-0.602 | 0.498 | 0.405-0.578 | 0.505 | 0.411-0.59 | 0.511 | ||
A*: A_EFS_All; B*: B_OS_All; C*: C_SEX_All; D*: D_FAV_All; E*: E_EFS_HR; F*: F_OS_HR; AUC: Area under ROC curve; CI: Confidence interval; 95% CI was calculated from the bootstrap estimation. The upper-right and lower-left regions are for the untrained models built using cross-platform transferred signature genes, while the upper-left and lower-right regions are for the models originally trained.