Misclassification of Self-Directed Violence

Abstract

Abstract: Background: Classification of acts of self-directed violence has been shown to be inadequate in past research. Furthermore, level of expertise have been shown to be unrelated to classification correctness. Aim: The aim of the present study was to investigate whether participants provided with a definition are more reliable in their judgment than participants without a definition. Method: Two hundred sixty-one participants (psychology students, psychotherapists-in-training) were presented with case vignettes describing different acts of self-directed violence and were asked to make a classification. On the basis of randomized allocation, half of the participants received a definition of the different acts of self-directed violence, whereas the others did not. Results: Overall, 24.9% of the cases were misclassified. The presentation of a definition was not accompanied by a higher classification accuracy. Limitations: There may be issues about the validity of the case vignettes. Conclusions: The results highlight the importance of more methodological training of psychologists regarding suicidal issues.

Classification of acts of self-directed violence (SDV) has been shown to be inadequate in past research. In a study by Cwik and Teismann (2017), participants were presented with case vignettes describing different acts of SDV and were asked to make a classification. Overall, 51.6% of the cases were misclassified. Misclassification was most common in case vignettes describing undetermined self-directed violence and suicide attempts. The level of expertise was almost unrelated to classification correctness; as such, psychotherapists, psychotherapists-in-training, and psychology students did not differ in misclassification rates. Taken together, the study findings are rather daunting – in particular since classification errors could fundamentally alter mental health professionals’ perception of risk in clinical practice. However, a limitation of the study by Cwik and Teismann (2017) is that they did not provide participants with a formal definition of the described acts of SDV. As there is no generally accepted nomenclature and classification scheme (Goodfellow et al., 2020), it cannot be ruled out that participants had different classification systems in mind, when working on the case vignettes. Therefore, the purpose of this study was to determine whether participants provided with a definition are more reliable in their judgment than participants without a definition. If using a definition results in a lower misclassification rate, this would underscore the need for widely disseminating a standard definition.

In order to establish a high degree of comparability with the study by Cwik and Teismann (2017), the same case vignettes (plus four more) and SDV definitions provided by the Center for Disease Control and Prevention (CDC; Crosby et al, 2011) were used: According to the CDC, suicide is defined as a “death caused by self-directed injurious behavior with any intent to die as a result of the behavior” (p. 23); the term suicide attempt refers to “a nonfatal self-directed potentially injurious behavior with any intent to die as a result of the behavior” (p. 21); the term nonsuicidal self-directed violence is used for “behavior that is self-directed and deliberately results in injury or the potential for injury to oneself”, while “there is no evidence, whether implicit or explicit, of suicidal intent” (p. 21); and the term “undetermined self-directed violence” is indicated when the intent is not ascertainable and cannot be inferred.

Method

Participants

In total, 270 participants took part in the study. Nine participants had to be excluded due to incomplete data. The final sample comprised N = 261 (83.1% female; M_age = 25.15, SD_age = 6.56) participants: n = 207 psychology students (79.3%) and n = 54 psychotherapists-in-training (20.7%). Participants were randomly assigned to one of two groups: n = 133 received a definition (Def; 85% female; 78.9% students; M_age = 25.64, SD_age = 6.73) and n = 128 received no definition (NoDef; 81.3% female; 79.1% students; M_age = 14.64, SD_age = 6.36). Groups did not differ regarding age, gender, and expertise level.

Procedure

An online survey questionnaire was composed with SoSci Survey. Social media postings were used to promote the study. Prior to assessments, participants were informed that the study was about the classification of acts of SDV. Furthermore, they were informed about the voluntary nature of their participation, data storage, and security. All participants gave their informed consent. The Ethics Committee of the Ruhr University Bochum approved the study.

Materials

Seventeen vignettes of SDV were drawn from the training material provided by Crosby et al. (2011, pp. 75–82). The vignettes are descriptions of persons either experiencing suicidal ideation or performing a specific form of SDV (suicide, suicide attempt, suicide attempt interrupted by self or by other, nonsuicidal self-directed violence, unintentional injury death, other suicidal behavior, not enough information/undetermined). Individuals were instructed to read the vignettes and to identify the most accurate CDC term associated with the thoughts and/or behaviors described. Participants in the Def group were provided with a formal definition of the different acts of SDV according to Crosby et al. (2011, pp. 21–23). Regarding suicidal ideation, a definition by Wenzel et al. (2009) was used: “Any thoughts, images, beliefs, voices, or other cognitions reported by the individual about intentionally ending his or her own life” (p. 18). The definitions were presented once before the vignettes were displayed.

Data Analyses

Data analysis was conducted using SPSS version 27.0 for Mac. Descriptive data are presented as frequencies (%). To investigate differences between groups (Def vs. NoDef), χ² tests were conducted. For this purpose, the nine response options to the case vignettes were dichotomized (correct vs. incorrect). Bonferroni correction was used to adjust the probability value due to multiple testing. The p-value for all group comparisons was set at p < .0029.

Results

The percentages of correct classification decisions of participants for the 17 case vignettes can be seen in Figure 1. Taken together, allocations of all participants were moderate, with a total correct classification rate of 75.1% and a misclassification rate of 24.9%. Correct classifications of psychology students ranged from 9.2% to 97.6% and of psychotherapists-in-training from 14.8% to 100%. Psychology students and psychotherapists-in-training did not differ in classification correctness, χ²(1) < 8.57, ns.

Figure 1. Percentages of correct diagnostic decisions of participants for all case vignettes.Note. Def = definition provided; NoDef = no definition provided; S = suicide; SI = suicide ideation; SA = suicide attempt; SA_IO = suicide attempt interrupted by others; SA_IS = suicide attempt interrupted by self; NEI = not enough information; NSSI = nonsuicidal self-directed violence; OSB = other suicidal behavior; UID = unintentional injury death.

Correct classifications of participants provided with a definition (Def) ranged from 10.5% to 98.5% and of participants without definition (NoDef) from 10.2% to 97.7% (see Figure 1). Participants provided with a definition scheme and those without did not differ in classification correctness, all χ²(1) < 2.65, ns.

Discussion

In the present study, we investigated the classification of SDV conducted by participants who were or were not provided with a definition of SDV terms. Overall, correct classification varied from 10.3% to 97.3%, with a mean classification rate of 75.1%. Misclassification was most common in cases that could not be classified due to missing information. A relatively low misclassification rate (< 5%) was found regarding vignettes that described a suicide or a suicide attempt interrupted by self, whereas suicide attempts were misclassified comparably often (cf. Cwik & Teismann, 2017). Of note, the misclassification rate was substantially lower in the present study than in the study by Cwik and Teismann (2017): 24.9% versus 51.6%. This fact remains even if the four additionally examined vignettes used in the present study were removed from the overall evaluation. The reasons for this improved classification are unclear, but it points to the possibility that knowledge about aspects of suicidal ideation and behavior has increased in recent years.

In line with previous studies, the level of expertise was unrelated to classification correctness. However, contrary to what was expected, asking raters to read SDV definitions before classifying the vignettes did not improve the accuracy of classification (cf. Wagner et al., 2002). In fact, the experimental groups did not differ in their classification accuracy with regard to type of SDV. The misclassification rate also did not differ by vignette. It might be that participants did not pay enough attention to the definitions because they mistakenly believed that recognizing a suicide or a suicide attempt is obvious and easy. It might also be that the provision of a definition did not sufficiently help participants to correctly classify difficult cases or to free themselves from pre-existing definitional ideas. The current study showed that providing a short definition before rating SDV types was not sufficient to enhance classification accuracy. Instead, active learning through training and practice may be necessary to increase classification accuracy of SDV. In this sense, studies in which trained raters are asked to perform classifications in real-world assessment situations show a significantly higher reliability in their ratings than participants in the present study (Mundt et al., 2010).

Limitations

Some limitations have to be considered when interpreting the current results. First, vignettes may not have adequate construct validity and/or generalizability. Furthermore, the validity of case vignettes compared to real-life settings is questionable (Evans et al., 2015). This said, the vignettes used in the current study included sufficient, clear, and unequivocal information. Second, no licensed psychotherapists were recruited for the current study. Therefore, it is unclear whether findings would also generalize to participants with greater expertise. However, in previous studies, a higher level of expertise was unrelated to classification accuracy (Cwik & Teismann, 2017). Third, the definitions were presented only once, and no examples were given to clarify the definitions. Future studies should investigate whether classifications accuracy can be increased if definitions are shown again with each new vignette and participants are provided with exercise examples in advance. Fourth, Crosby et al.'s (2011) vignettes used in the present study have neither undergone an evaluation of validity nor are they constructed in such a way that only individual defining characteristics are varied while all other descriptive features are held constant. Accordingly, no conclusions can be drawn as to which components of a definition make classification particularly difficult or easy. This would need to be addressed in a future study. Finally, the results may have been different if alternative definitions of SDV had been used (Goodfellow et al., 2020).

Conclusion

Taken together, the study results indicate that providing raters with definitions of acts of SDV does not improve reliability of their ratings of hypothetical clinical situations (with limited information).

Biographies

Author Biographies

Tobias Teismann, PhD, is a psychologist and head of the Center for Psychotherapy at the Ruhr University Bochum, Germany. He leads the Suicidal Behavior Research Group.

Jannik Julian Eimen, MSc, is a psychologist. He wrote his master thesis in the field of suicidology.

Jan Christopher Cwik, PhD, is a psychologist. His main research interests are clinical assessment and diagnostic accuracy, psycho-oncology, and trauma-related disorders.

Publication Ethics: The study was approved by the Ethics Committee of the Faculty of Psychology at the Ruhr University Bochum, Germany.

Authorship: All authors have agreed to authorship.

Funding Statement

Funding The study was not funded by any granting agency. Open access publication enabled by Ruhr University Bochum, Germany.