Skip to main content
. 2020 Aug 26;78(1):77–90. doi: 10.1001/jamapsychiatry.2020.2459

Table 2. Performance Metrics for Unadjusted Support Vector Machine Models.

Model description Transition, No./total No. (%) Nontransition, No./total No. (%) Sensitivity, % Specificity, % Balanced accuracy, % AUC (95% CI) PPV, % NPV, % Positive likelihood ratio Negative likelihood ratio
True positive False negative True negative False positive

  • Model 1a: EU-GEI clinical and proteomic dataa

  • Data set: EU-GEI initial experiment, all sites

  • Features: 69 clinical and 166 proteomic

  • Target: transition status

  • N: 49 transition, 84 nontransition

 48/49 (98.0) 1/49 (2.0) 68/84 (81.0) 16/84 (19.0) 98.0 81.0 89.5 0.95 (0.91-0.99) 75.0 98.6 5.1 <0.1

  • Model 1b: EU-GEI clinical dataa

  • Data set: EU-GEI initial experiment, all sites

  • Features: 69 clinical

  • Target: transition status

  • N: 49 transition, 84 nontransition

 23/49 (46.9) 26/49 (53.1) 45/84 (53.6) 39/84 (46.4) 46.9 53.6 50.3 0.48 (0.38-0.58) 37.1 63.4 1.0 1.0

  • Model 1c: EU-GEI proteomic dataa

  • Data set: EU-GEI initial experiment, all sites

  • Features: 166 proteomic

  • Target: transition status

  • N: 49 transition, 84 nontransition

 49/49 (100) 0/49 (0) 71/84 (84.5) 13/84 (15.5) 100 84.5 92.3 0.96 (0.92-1.00) 79.0 100 6.5 <0.1

  • Model 2a: EU-GEI proteomic data (non-London)a

  • Data set: EU-GEI initial experiment, all sites except London

  • Features: 166 proteomic

  • Target: transition status

  • N: 30 transition, 50 nontransition

 28/30 (93.3) 2/30 (6.7) 40/50 (80.0) 10/50 (20.0) 93.3 80.0 86.7 0.94 (0.88-1.00) 73.7 95.2 4.7 0.1

  • Model 2b: top 10, training dataa

  • Data set: EU-GEI initial experiment, all sites except London

  • Features: 10 proteomic

  • Target: transition status

  • N: 30 transition, 50 nontransition

 30/30 (100) 0/30 41/50 (82.0) 9/50 (18.0) 100 82.0 91.0 0.99 (0.96-1.00) 76.9 100 5.6 <0.1

  • Model 2b: top 10, test dataa

  • Data set: EU-GEI initial experiment, London site

  • Features: 10 proteomic

  • Target: transition status

  • N: 19 transition, 34 nontransition

 18/19 (94.7) 1/19 (5.3) 30/34 (88.2) 4/34 (11.8) 94.7 88.2 91.5 0.92 (0.83-1.00) 81.8 96.8 8.1 0.1

  • Model 3: EU-GEI clinical and proteomic dataa

  • Data set: EU-GEI replication experiment, all sites

  • Features: 69 clinical and 119 proteomic

  • Target: transition status

  • N: 49 transition, 86 nontransition

 48/49 (98.0) 1/49 (2.0) 77/86 (89.5) 9/86 (10.5) 98.0 89.5 93.7 0.98 (0.95-1.00) 84.2 98.7 9.4 <0.1

  • Model 4: ALSPAC proteomic dataa

  • Data set: ALSPAC

  • Features: 265 proteomic

  • Target: PEs at age 18 y

  • N: 55 PEs, 66 no PE

 40/55 (72.7) 15/55 (27.3) 47/66 (71.2) 19/66 (28.8) 72.7 71.2 72.0 0.74 (0.65-0.83) 67.8 75.8 2.5 0.4

  • Model S1: ELISA

  • Data set: EU-GEI initial experiment, all sites

  • Features: 9 ELISA

  • Target: transition status

  • N: 44 transition, 82 nontransition

 33/44 (75.0) 11/44 (25.0) 51/82 (62.2) 31/82 (37.8) 75.0 62.2 68.6 0.76 (0.67-0.85) 51.6 82.3 2.0 0.4

  • Model S2: functional outcome

  • Data set: EU-GEI initial experiment, all sites

  • Features: 69 clinical and 166 proteomic

  • Target: functional outcome

  • N: 47 poor functioning (GAF ≤60); 32 good functioning (GAF >60)

 27/47 (57.4) 20/47 (42.6) 22/32 (68.8) 10/32 (31.3) 57.4 68.8 63.1 0.74 (0.63-0.85) 73.0 52.4 1.8 0.6

Abbreviations: ALSPAC, Avon Longitudinal Study of Parents and Children; AUC, area under the receiver operating characteristic curve; ELISA, enzyme-linked immunosorbent assay; EU-GEI, European Network of National Schizophrenia Networks Studying Gene-Environment Interactions; NPV, negative predictive value; PE, psychotic experience; PPV, positive predictive value.

a

Models 1a-c, 2, and 3 are adjusted for age, sex, body mass index, and years of education, and model 4 is additionally adjusted for race/ethnicity and tobacco use.