Table 2.
F1-score evaluation of the detected helices and β-sheets in 62 box-cropped map/structure pairs for the CE, FL, DL, and FL_DL loss functions.
| EMDB_PDB_Chain (resolution in Å) | Count (H/S/O) | F1 |
F1 |
F1 |
F1 |
Residue-level F1 |
|||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CE (%) |
FL (%) |
DL (%) |
FL_DL (%) |
FL_DL (%) |
|||||||
| H-V | S-V | H-V | S-V | H-V | S-V | H-V | S-V | H-R | S-R | ||
| 0090_6GYK_M (5.1) | 557/324/648 | 53.6 | 41.6 | 56.2 | 26.8 | 51.1 | 28.3 | 57.9 | 48.8 | 65.6 | 56.1 |
| 1657_4V5H_AE (5.8) | 158/68/229 | 58.1 | 50.3 | 61.7 | 53.5 | 59.1 | 51.9 | 60.1 | 46.9 | 69.4 | 59.6 |
| 1657_4V5H_AM (5.8) | 131/34/213 | 59.7 | 47.5 | 56.9 | 54.4 | 53.7 | 48.4 | 53.1 | 41.1 | 60.3 | 48.3 |
| 1798_4V5M_AE (7.8) | 164/95/151 | 47.1 | 37.4 | 50.1 | 48.3 | 50.7 | 41.5 | 48.8 | 47.2 | 55.1 | 57.0 |
| 2422_4V8Z_BZ (6.6) | 164/95/201 | 60.4 | 46.0 | 58.5 | 47.5 | 54.6 | 60.0 | 55.5 | 58.3 | 72.0 | 65.9 |
| 2594_4CR2_3 (7.7) | 480/335/398 | 65.5 | 51.6 | 63.3 | 58.2 | 55.0 | 42.3 | 58.0 | 55.8 | 81.4 | 61.5 |
| 2594_4CR2_S (7.7) | 786/33/189 | 66.7 | 0.4 | 64.9 | 1.4 | 67.7 | 20.0 | 67.5 | 0.7 | 82.1 | 0.0 |
| 2620_4UJE_BH (6.9) | 213/86/161 | 57.1 | 42.9 | 60.8 | 55.5 | 62.2 | 61.2 | 62.5 | 66.5 | 77.3 | 76.2 |
| 2620_4UJE_CL (6.9) | 322/91/372 | 55.3 | 28.5 | 60.5 | 40.8 | 62.9 | 32.7 | 62.1 | 51.9 | 72.4 | 68.0 |
| 2917_5AKA_O (5.7) | 66/7/329 | 46.2 | 12.0 | 44.9 | 6.9 | 51.6 | 11.1 | 52.3 | 11.0 | 65.5 | 19.0 |
| 3491_5MDX_H (5.3) | 296/6/149 | 59.7 | 0.0 | 59.9 | 0.0 | 54.0 | 0.0 | 56.9 | 3.5 | 63.8 | 0.0 |
| 3580_5MY1_I (7.6) | 168/45/262 | 24.9 | 21.7 | 31.4 | 24.3 | 36.0 | 36.2 | 32.7 | 30.3 | 36.0 | 35.8 |
| 3581_5MYJ_AK (5.6) | 75/27/124 | 45.4 | 54.3 | 42.2 | 57.4 | 42.1 | 65.1 | 3.4 | 51.3 | 0.0 | 65.6 |
| 3594_5N61_E (6.9) | 380/78/259 | 50.1 | 4.9 | 46.4 | 14.3 | 60.0 | 22.8 | 51.0 | 37.3 | 59.7 | 45.1 |
| 3663_5NO4_H (5.16) | 80/79/119 | 63.2 | 41.2 | 63.8 | 52.7 | 61.3 | 52.4 | 62.2 | 54.3 | 77.5 | 57.6 |
| 3850_5OQM_4 (5.8) | 509/136/367 | 57.8 | 44.0 | 57.6 | 53.0 | 60.4 | 45.6 | 58.2 | 51.7 | 68.6 | 62.3 |
| 3850_5OQM_g (5.8) | 468/126/231 | 73.9 | 42.4 | 74.6 | 36.7 | 70.2 | 48.0 | 74.7 | 51.4 | 91.8 | 59.1 |
| 3948_6ESG_B (5.4) | 259/15/119 | 65.1 | 0.0 | 65.4 | 25.1 | 69.9 | 34.0 | 67.0 | 24.9 | 76.3 | 35.0 |
| 4041_5LDX_H (5.6) | 775/49/647 | 70.6 | 53.4 | 72.3 | 53.9 | 69.4 | 46.9 | 70.9 | 46.3 | 85.7 | 57.1 |
| 4041_5LDX_I (5.6) | 585/175/1015 | 67.8 | 46.7 | 69.0 | 51.1 | 65.5 | 46.4 | 68.3 | 48.3 | 82.8 | 55.9 |
| 4075_5LMP_Q (5.35) | 71/59/70 | 72.9 | 45.9 | 67.4 | 59.1 | 71.5 | 71.2 | 72.7 | 65.7 | 82.4 | 76.6 |
| 4078_5LMS_D (5.1) | 154/63/172 | 62.5 | 32.9 | 65.2 | 55.4 | 65.1 | 57.4 | 70.2 | 53.0 | 81.1 | 59.7 |
| 4089_5LN3_D (6.8) | 582/214/453 | 67.6 | 54.1 | 66.2 | 56.5 | 65.3 | 56.0 | 69.0 | 59.2 | 79.4 | 63.8 |
| 4089_5LN3_G (6.8) | 596/279/453 | 70.9 | 44.3 | 66.4 | 53.4 | 65.8 | 49.9 | 68.6 | 54.1 | 80.7 | 55.7 |
| 4100_5LQX_H (7.9) | 282/65/326 | 71.0 | 55.0 | 68.6 | 61.0 | 65.9 | 61.1 | 61.0 | 58.1 | 74.3 | 72.4 |
| 4107_5LUF_M (9.1) | 1116/10/253 | 45.1 | 3.9 | 53.6 | 0.3 | 44.8 | 1.9 | 46.9 | 1.3 | 58.7 | 2.1 |
| 4141_5M1S_B (6.7) | 318/271/337 | 54.5 | 27.9 | 50.6 | 19.5 | 58.5 | 36.3 | 58.4 | 44.7 | 69.9 | 50.6 |
| 4177_6F38_A (6.7) | 582/160/319 | 67.3 | 48.1 | 68.3 | 53.7 | 68.0 | 59.6 | 69.2 | 54.8 | 82.3 | 66.5 |
| 4182_6F42_G (5.5) | 280/90/473 | 57.8 | 7.4 | 55.3 | 18.7 | 58.4 | 17.7 | 55.4 | 28.1 | 67.3 | 32.3 |
| 5030_4V68_B7 (6.4) | 31/0/77 | 14.9 | 0.3 | 31.0 | 0.0 | 30.6 | 0.3 | 32.8 | 1.9 | 35.3 | NA |
| 5036_4V69_AD (6.7) | 133/34/193 | 51.3 | 12.6 | 53.8 | 46.4 | 55.5 | 43.5 | 54 | 35.8 | 67.1 | 42.5 |
| 5942_3J6X_20 (6.1) | 65/61/124 | 53.3 | 44.3 | 54.9 | 45.8 | 53.3 | 51.5 | 43.8 | 40.1 | 51.9 | 47.1 |
| 5942_3J6X_25 (6.1) | 58/4/87 | 48.2 | 0.0 | 44.1 | 35.5 | 57.0 | 16.6 | 34.5 | 24.8 | 39.3 | 44.4 |
| 5942_3J6X_83 (6.1) | 120/52/157 | 48.8 | 29.3 | 65.4 | 54.9 | 62.9 | 26.3 | 68.5 | 57.0 | 83.9 | 81.3 |
| 5943_3J6Y_80 (6.1) | 34/28/76 | 62.7 | 54.9 | 61.2 | 56.6 | 65.8 | 43.6 | 54.6 | 54.9 | 59.1 | 65.3 |
| 6149_3J8G_W (5.0) | 22/59/58 | 9.9 | 18.3 | 29.5 | 55.0 | 24.6 | 57.6 | 14.9 | 61.3 | 9.5 | 82.4 |
| 6446_3JBI_V (8.5) | 414/134/213 | 57.7 | 27.2 | 58.5 | 37.8 | 55.8 | 45.4 | 53.7 | 41.9 | 72.6 | 55.0 |
| 6452_3JBO_AH (5.8) | 210/157/264 | 67.6 | 56.8 | 68.6 | 51.9 | 69.2 | 48.8 | 70.5 | 49.7 | 81.1 | 56.4 |
| 6452_3JBO_AS (5.8) | 358/103/398 | 46.5 | 24.8 | 61.2 | 50.7 | 63.4 | 26.7 | 61.7 | 38.2 | 67.1 | 58.1 |
| 6452_3JBO_AZ (5.8) | 292/95/200 | 62.6 | 52.1 | 67.6 | 60.5 | 67.7 | 55.0 | 67.7 | 41.1 | 78.1 | 62.3 |
| 6452_3JBO_Ae (5.8) | 162/66/186 | 54.4 | 33.1 | 66.5 | 47.3 | 65.1 | 31.9 | 66.0 | 37.6 | 75.9 | 43.5 |
| 6452_3JBO_C (5.8) | 320/154/331 | 63.5 | 63.2 | 64.8 | 67.9 | 65.6 | 61.0 | 63.0 | 63.5 | 72.0 | 73.9 |
| 6456_3JBN_AL (6.7) | 389/94/341 | 54.2 | 24.3 | 59.5 | 44.9 | 60.1 | 21.1 | 59.3 | 34.1 | 69.4 | 51.0 |
| 6456_3JBN_AP (6.7) | 360/64/375 | 54.6 | 13.4 | 59.9 | 27.5 | 62.6 | 11.3 | 60.6 | 25.7 | 68.6 | 66.7 |
| 6456_3JBN_U (6.7) | 442/203/390 | 70.3 | 62.6 | 71.3 | 61.8 | 69.3 | 53.8 | 69.9 | 57.6 | 81.9 | 65.1 |
| 6585_5IMR_p (5.9) | 70/18/95 | 65.2 | 46.3 | 66.6 | 56.6 | 61.8 | 50.6 | 65.3 | 50.6 | 83.3 | 73.9 |
| 6585_5IMR_u (5.9) | 46/6/38 | 33.1 | 35.2 | 66.4 | 25.3 | 65.0 | 35.2 | 54.2 | 22.8 | 60.0 | 44.4 |
| 6810_5Y5X_H (5.0) | 254/12/188 | 51.0 | 2.7 | 61.0 | 5.8 | 51.0 | 9.5 | 61.8 | 7.9 | 73.9 | 11.3 |
| 7454_6D84_S (6.72) | 334/43/127 | 49.2 | 22.8 | 43.9 | 30.3 | 52.1 | 27.3 | 44.3 | 29.5 | 50.4 | 40.0 |
| 8016_5GAR_O (6.4) | 366/0/148 | 60.0 | NA | 59.6 | NA | 57.5 | NA | 55.6 | NA | 63.5 | NA |
| 8128_5J7Y_K (6.7) | 965/17/271 | 76.2 | 51.1 | 74.7 | 58.1 | 73.3 | 38.6 | 73.9 | 61.6 | 89.3 | 85.1 |
| 8129_5J8K_AA (6.7) | 664/95/572 | 61.8 | 41.9 | 62.0 | 47.9 | 61.4 | 36.2 | 60.4 | 45.0 | 73.3 | 53.5 |
| 8129_5J8K_D (6.7) | 830/96/995 | 60.6 | 36.1 | 59.2 | 41.4 | 58.0 | 33.3 | 58.3 | 45.2 | 72.4 | 56.9 |
| 8130_5J4Z_B (5.8) | 656/88/805 | 74.2 | 58.7 | 73.5 | 53.6 | 68.5 | 45.2 | 72.2 | 55.1 | 86.6 | 69.5 |
| 8135_5IYA_E (5.4) | 319/91/256 | 62.8 | 29.1 | 61.7 | 41.2 | 63.1 | 36.4 | 63.0 | 34.8 | 73.4 | 38.5 |
| 8335_5T0H_K (6.8) | 221/78/221 | 57.4 | 44.4 | 62.1 | 51.1 | 64.4 | 53.3 | 61.1 | 54.8 | 69.6 | 67.5 |
| 8357_5T4O_L (6.9) | 273/107/458 | 72.3 | 56.0 | 71.0 | 52.9 | 66.7 | 52.6 | 65.5 | 55.5 | 78.6 | 67.5 |
| 8518_5U8S_2 (6.1) | 612/240/840 | 62.9 | 48.3 | 64.2 | 51.1 | 61.2 | 48.2 | 61.7 | 48.0 | 78.6 | 56.3 |
| 8518_5U8S_A (6.1) | 529/43/343 | 76.2 | 40.2 | 74.9 | 39.3 | 72.4 | 44.8 | 73.2 | 41.1 | 88.2 | 47.1 |
| 8621_5UZ4_J (5.8) | 121/69/232 | 50.8 | 26.4 | 48.4 | 38.5 | 53.8 | 47.6 | 49.9 | 50.1 | 56.1 | 53.3 |
| 8693_5VIY_A (6.2) | 323/299/485 | 58.4 | 28.3 | 62.2 | 43.1 | 57.7 | 40.2 | 64.9 | 56.4 | 80.5 | 67.0 |
| 9534_5GPN_Ae (5.4) | 114/10/64 | 58.3 | 46.4 | 56.6 | 47.3 | 59.5 | 52.4 | 55.3 | 44.6 | 69.6 | 66.7 |
| Weighted Average | 20 724/5735/18 647 | 61.7 | 39.9 | 62.9 | 45.4 | 61.8 | 43.4 | 62.0 | 48.7 | 74.3 | 58.2 |
The center box of each cropped map indicates the EMDB ID, the PDB ID, the author-annotated chain ID, the resolution of the cryo-EM map, and the number of Cα atoms in the center box obtained from the author-annotated CIF file. Voxel-level F1-scores (in percent) are shown for helix and β-sheet detection in the center box when CE, FL, DL, and FL_DL were used to train the CNN models. In each case, the highest score for helix and β-sheet detection is highlighted in bold for each case. Residue-level F1-scores are shown for the FL_DL loss function. The weighted average of F1-scores over the 62 test cases was calculated using the number of voxels of helix/β-sheet/background in each class as weights. H, helix; S, β-sheet; O, other/background; NA, no voxel or residue existent (undefined F1-score).