Table 5. An ablation study performed on Throwaway accounts (TA; User types), Supportive (SU), and Un-Informative(UI) Posts (Content-types) to evaluate the performance of suicide risk assessment frameworks in Time-invariant (a and b) and Time-variant (c and d) settings.
In the TinvM context, irrespective of user-type, all types of content are required for high precision and high recall in predicting user-level suicidality. Lengthy posts expressing mental health conditions are often made by TA (a), which resulted in high precision compared to Non-TA (b). However, in the TvarM, seldom supportive behavior of suicidal users is important for assessing their suicidality (c). For Non-TA, there is a trade-off between precision and recall concerning uninformative posts. Still, supportive posts help determine the severity of an individual’s suicide risk (d). For clinical-grounding based assessment, we recorded the results in Table 7.
| SNo. | TA | UI | SU | Avg. Prec. | Avg. Rec. | F1 | SNo. | TA | UI | SU | Avg. Prec. | Avg. Rec. | F1 |
| S1 | yes | yes | yes | 0.70 | 0.58 | 0.63 | S5 | no | yes | yes | 0.64 | 0.56 | 0.60 |
| S2 | yes | yes | no | 0.58 | 0.48 | 0.52 | S6 | no | yes | no | 0.59 | 0.50 | 0.55 |
| S3 | yes | no | yes | 0.44 | 0.49 | 0.46 | S7 | no | no | yes | 0.45 | 0.42 | 0.43 |
| S4 | yes | no | no | 0.55 | 0.50 | 0.53 | S8 | no | no | no | 0.55 | 0.53 | 0.54 |
| (a) TinvM with Throwaway Accounts | (b) TinvM without Throwaway Accounts | ||||||||||||
| S9 | yes | yes | yes | 0.71 | 0.66 | 0.68 | S13 | no | yes | yes | 0.80 | 0.65 | 0.76 |
| S10 | yes | yes | no | 1.0 | 0.45 | 0.62 | S14 | no | yes | no | 1.0 | 0.34 | 0.50 |
| S11 | yes | no | yes | 1.0 | 0.66 | 0.79 | S15 | no | no | yes | 1.0 | 0.63 | 0.77 |
| S12 | yes | no | no | 1.0 | 0.49 | 0.66 | S16 | no | no | no | 1.0 | 0.50 | 0.66 |
| (c) TvarM with Throwaway Accounts | (d) TvarM without Throwaway Accounts | ||||||||||||