Psychometric Properties of the Suicidal Ideation Scale in the Indonesian Language

Abstract

Introduction:

Suicide is one of the leading cause of mortality among adolescents and young adults, especially in low to middle-income countries. Research found that screening for suicidal ideation in non-clinical populations such as schools or communities is an important step toward preventing suicide. Despite so, not all screening tools have the capabilities to accurately detect suicidal ideation among adolescents and young adults in non-clinical populations. The Suicidal Ideation Scale (SIS) is one of the most used questionnaires to measure suicidal thoughts in non-clinical populations. This study aims to investigate the psychometric properties of SIS among adolescents and young adults in Indonesia, especially in non-clinical populations.

Methods:

After a series of language and cultural adaptations, 1254 senior high school and university students completed the Indonesian version of SIS using 3 standard questionnaires, namely Patient Health Questionnaire-9/PHQ-9, Beck Depression Inventory-II/BDI-II, and Children’s Depression Inventory/CDI. The SIS content validity, internal consistency, test-retest reliability and concurrent, as well as internal structure validity, were investigated using content validity index (CVI), Cronbach’s Alpha, Pearson product-moment correlation, and confirmatory factor analysis (CFA), respectively.

Results:

The result showed that SIS has good to excellent internal consistency and test-retest reliability. Based on the validity indicators, it has satisfactory content and convergent validity, and further support the one-factor and 2-factor model for factorial validity. Both one-factor and 2 factor model are suitable to use in non-clinical settings.

Conclusions:

SIS is a valid and reliable tool for suicide ideation screening in adolescents and young adults in non-clinical populations. This validated questionnaire can be used in the early detection of suicidal ideation among adolescents and young adults in non-clinical populations, thus contributing to developing strategies and policies to prevent suicide among Indonesian adolescents and young adults at group and institutional levels.

Introduction

Suicide is an act of hurting oneself deliberately to cause death and is the fourth leading cause of mortality among 15 to 29 years old, especially in low to middle-income countries (LMICs).^1,2 Currently, the prevalence of suicide in adolescents and young adults is increasing. This might be due to several reasons, such as higher time spent on media and decreases in physical activity, which eventually lead to depression and suicide.³ Another reason is the present suicide prevention effort tends to focus on an individual-level approach rather than a group or institutional approach.⁴ In Indonesia, the rate is 3.7 per 100 000 population, where 4.75% of adolescents (13-18 years old) had suicidal ideas and 2.46% had attempted suicide in 2015.^1,2 A previous study reported that 18.3% of adolescents have suicidal ideas in Jakarta.⁵

This suicide phenomenon is usually developed and progresses from suicidal ideation into behavior. Suicidal ideation or thoughts is a broad term used to describe a range of contemplations, wishes, and preoccupations with death and suicide.⁶ Research found that screening for suicidal ideation in non-clinical populations such as schools or communities is an important step toward preventing suicide.⁷ Despite so, a systematic review by the U.S. Preventive Services Task Force (USPSTF) showed that not all screening tools have the capabilities to accurately detect suicidal ideation among adolescents and young adults in non-clinical populations.⁸ Consequently, a screening tool to identify suicidal ideation among adolescents and young adults in non-clinical settings is needed.

The Suicidal Ideation Scale (SIS) was developed as a short instrument to assess the severity of suicidal behavior in non-clinical populations.⁹ It was conceptualized to represent a continuum of suicidal ideation ranging from covert thoughts to more overt or intense ideation and actual suicide attempts. SIS consists of 10 items, divided into 2 factors, namely (i) suicide desire (ongoing thoughts) and (ii) resolved plans and preparation (intense thought, plans, courage, and capability to attempt suicide). It was shown that SIS has a high level of internal consistency (Cronbach Alpha = .86) and adequate item-total correlations (r = .45-.74).⁹ Luxton et al¹⁰ discovered higher internal consistency (Cronbach alpha = .91) and higher correlations items to scale ranged from .57 to .89.

Since its development, SIS has been translated and validated in several countries. Han et al¹¹validated SIS in South Korea among patients with mental disorders using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). The results showed that SIS has excellent psychometric properties, including internal consistency reliability (with .91 Cronbach’s alpha coefficient), convergent validity, and known-group validity. It was also shown that SIS is more comprehensible to students and able to detect early to serious suicidal ideation. However, no published report about SIS validation in Indonesia can be found in the literature.

Therefore, this study aims to adapt the Suicidal Ideation Scale to Bahasa Indonesia and investigate the psychometric properties among adolescents and young adults. The objective of this study is to determine the internal consistency reliability and validity of the SIS which involve content validity, convergent validity, and factorial validity among Indonesian adolescents and young adults in non-clinical populations. This validated questionnaire can be used in the early detection of suicidal ideation, in non-clinical populations, thus contributing to develop strategies and policies to prevent suicide among adolescents and young adults at group and institutional level.

Methods

Design and Participants

This research used a cross-sectional survey study with a quantitative method research approach. The sampling technique used consecutive sampling.¹² The minimum sample size was determined by following the 10-times rule, which recommends that a sample size should be equal to the larger of 10 times the largest number of formative indicators used to measure one construct.¹³ By following the 10-times rule, a total of 270 minimum sample size was determined. This study involved 344 senior high school and 910 university students, respectively in Bandung, West Java Province, Indonesia. The ages of high school students range between 15 and 18 and the ages of university students range between 18 and 25. The characteristics of participants is explained in Table 1.

Table 1.

Characteristics of Participants.

Characteristics	n (%)
Age
Mean (SD)	19.39 (1.79)
Range	15-25
Gender
Female	920 (73)
Male	334 (27)
Education level
Senior high school	334 (27)
University	910 (73)
Religion
Islam	1078 (86)
Christian	164 (13)
Other (Agnostic, Hindu, Buddha)	12 (1)
Marital status
Single	1253 (99.9)
Married	1 (0.1)

Measures

Suicidal Ideation Scale (SIS)

The Suicide Ideation Scale (SIS) is a 10-item self-report questionnaire identifying ideation that differentiates covert suicidal thoughts from more overt or intense ideaion.⁹ The response options are Likert-type scale from 1 (never or none of the time) to 5 (always or a great many times) based on how the participant has felt or behaved in the past week. SIS consists of 2 subscales, suicidal desire which involve of 4 items and resolved plans and preparations which involve 6 items (as seen in Table 2).¹⁰ Furthermore, it has shown high internal consistency (Cronbach Alpha = .86) and an adequate item-total correlations (rs = .45-.74).⁹ Luxton et al discovered higher internal consistency of Cronbach Alpha = .91, and correlations items to scale ranged from .57 to .89.¹⁰

Table 2.

SIS Subscale.

Subscale	Item number (based on to original SIS)	Item
Suicidal desire	7	I just wish my life would end
	5	I feel life just isn’t worth living
	6	Life is so bad I feel like giving up
	8	It would be better for everyone involved if I were to die
Resolved plans and preparations	10	I have come close to taking my own life
	4	I have made attempts to kill myself
	1	I have been thinking of ways to kill myself
	3	I believe my life will end in suicide
	9	I feel there is no solution to my problems other than taking my own life
	2	I have told someone I want to kill myself

Patient Health Questionnaire-9 (PHQ-9)

The Patient Health Questionnaire-9 (PHQ-9) consisted of 9 questions for screening, diagnosing, monitoring, and measuring the severity of depression. The response options ranged from not at all (0), several days (1), more than half of the days (2), and nearly every day (3). An example item of PHQ-9 is “Thoughts that you would be better off dead, or hurting yourself.” Previous investigations using the English and Indonesian versions discovered that PHQ-9 reliability scores ranged from 0.73 to 0.89,^14-16 while PHQ-9’s Cronbach Alpha coefficient of .88 was obtained in this study.

Beck Depression Inventory-II (BDI-II)

The Beck et al¹⁷ Depression Inventory-II (BDI-II) is a 21-item self-administered inventory to assess the severity of depressive symptoms. Participants were asked to rate each item based on 4 response options according to the severity of the symptoms. This ranged from the absence of a symptom to an intense level during the past week and each response option varies from 0 (not) to 3 (severe). An example item of BDI-II is suicidal thoughts or wishes such as “I don’t have any thoughts of killing myself” for option score 0 to “I would kill myself if I had the chance” for option score 3. The test covers cognitive, which was thoughts about past failure, emotional/affective, sadness, and somatic/vegetative, namely tiredness or fatigue symptoms.¹⁷ Beck et al¹⁸ and Wang and Gorenstein,¹⁹ reported that the use of BDI-II in various languages with a mean Cronbach’s Alpha of .9 obtained a value ranging from .83 to .96, and the Indonesian version had been validated.²⁰ In this study, the Cronbach Alpha coefficient is .94, although only the university students completed the BDI-II.

Children’s Depression Inventory (CDI)

The Children’s Depression Inventory (CDI) consists of 27 items to assess symptoms of depression experienced in the past 2 weeks.²¹ Participants select 1 out of the 3 alternative statements for each symptom such as “I don’t think about committing suicide”; “I think about committing suicide”; or “I want to commit suicide.” Each statement scored 0, 1, or 2 points, with higher scores indicating greater levels of depression. CDI has established good reliability for describing depressive symptoms in Asian populations.^22,23 In this study, only the senior high school students completed the CDI and the Cronbach Alpha coefficient is .86.

Procedure

The approval to translate and validate SIS was obtained from M. David Rudd, PhD., ABPP on 3 May 2020, and the adaptation process followed a standardized guideline.²⁴ Firstly, the original version of SIS was translated from English into Indonesia by 2 professional translators, followed by a discussion leading to a synthesized version. This was followed by the back-translation of the synthesized Indonesian version into English by 2 other bilinguals. Thirdly, the back-translated version was compared with the original version of SIS. This comparison was discussed by a committee of 5 experts from departments of psychiatry, psychology, public health, and clinicians. Any discrepancies found were discussed and changed by the committee. Fourthly, several cognitive interview sessions with high school and university students were conducted to obtain the final version. Indonesian version of SIS can be seen in Table 3.

Table 3.

Indonesian Version of SIS.

Item number	Item
1	Saya memikirkan berbagai cara untuk bunuh diri.
2	Saya pernah mengatakan pada seseorang kalau saya ingin bunuh diri
3	Saya percaya hidup saya akan berakhir dengan bunuh diri
4	Saya pernah mencoba untuk bunuh diri
5	Saya merasa bahwa hidup sudah tidak berarti lagi
6	Hidup ini begitu buruk, saya merasa ingin menyerah
7	Saya berharap hidup saya berakhir
8	Akan lebih baik bagi orang-orang dalam hidup saya jika saya mati
9	Saya merasa tidak ada solusi dari masalah saya selain dengan mengakhiri hidup saya sendiri.
10	Saya nyaris mengakhiri hidup saya sendiri

Data for this validation study was collected online via the SurveyMonkey app in a Zoom video conference platform. This was carried out to ensure any difficulties and unwanted effects that can happen during the completion of the questionnaires were solved. All participants signed informed consent and followed by completing the Suicidal Ideation Scales (SIS), the Patient Health Questionnaire (PHQ), the Beck Depression Inventory-II (BDI-II), and the Children Depression Inventory (CDI) for adolescents. Furthermore, participants completed a retest about 1 until 2 weeks after the first data collection.

Data Analysis

Reliability Analysis

Alpha Cronbach reliability test was used to measure the internal consistency and test re-test was also used to determine reliability. The Alpha Cronbach’s coefficient is reliable if the value is .6 or higher.²⁵ The test-retest reliability coefficient was calculated using Pearson product-moment correlation between the first and the second time measure, where a correlation of .1 to .3 is considered weak, .4 to .6 is moderate, .7 to .9 is strong, and 1.0 is perfect.²⁶

Content Validity

The content validity analysis was carried out to determine if the language content, and structure of the new instrument (Indonesian version) was adequate for measuring suicidal ideation. The content validity index (CVI) consists of 2 index: the item-level content validity index (I-CVI) determining the content validity of individual items and the scale-level content validity index (S-CVI) determining the content validity of an overall scale. The proportion of .78 or above for the content validity for individual items is considered an acceptable content validity.²⁷ The acceptable scores for content validity of an overall scale are between .8 and .9 or greater.²⁸ Content validity evidence of SIS was measured by the content validity index (CVI), based on expert judgments.

Convergent Validity

Convergent evidence shows that measures of the same construct converge, or narrow in, on the same thing. When a measure correlates well with other tests believed to measure the same construct, convergent evidence for validity is obtained. In many ways, convergent evidence that is also construct validity evidence is like criterion validity evidence. In each case, scores on the test are relates to scores on some other measure. In the case of convergent evidence for construct-related validity, however, there is no criterion to define what we are attempting to measure.

This study used BDI-II as a convergent evidence for construct related validity.²⁹ Beck depression inventory II (BDI II) is used for adolescents and children depression inventory (CDI) for children. PHQ-9 is used as a convergent evidence, this criterion is based on the research by Na et al³⁰ in 2018. Pearson Product Moment correlation is applied to test the relation between SIS and BDI, CDI and PHQ-9.

Factorial Validity

The internal structure of SIS was evaluated by the confirmatory factor analysis (CFA) using LISREL 10.3 and the factor model being analyzed was based on the discovery of Luxton et al.¹⁰ CFA overcomes several limitations inherent in the exploratory factor model and is used for testing the best fit of a priori-defined model.³¹ The model was assessed using the adjusted goodness-of-fit index (AGFI), comparative fit index (CFI), the non-normed fit index (NNFI), root mean squared error of approximation (RMSEA), and standardized root mean square residual (SRMR). AGFI, CFI, and NNFI values range from 0 to 1.0, with values >0.9, indicating a good fit to the data. Meanwhile, smaller values of RMSEA and SRMR indicate a better fit, with values <0.10 implying a good fit, and values <0.05 consider a very good fit.³²

Mean and standard deviation was computed to describe the data and the number of participants that were categorized as having serious suicidal ideation was based on both statistics. The T-test was used to describe the relation of suicidal ideation toward gender and education group. Cut-off point to categorize serious suicidal ideation was based on the score of mean plus one standard deviation (SD).

Result

Descriptive Statistics of SIS

Out of the 1254 students, the suicidal ideation scores ranged from 10 to 46, with a mean (M) of 15.27 and a standard deviation (SD) of 6.62 and the prevalence rate was 15%. The relation between SIS scores with gender and educational background is shown in Table 4. Based on the results, a significant difference in SIS total score and suicidal desire subscale score between genders was discovered. The 2 subscale scores were significantly different between senior high school and university students.

Table 4.

Mean, Standard Deviation, and the t-test Comparison.

SIS and subscales	Gender and education	M (SD)	t(df)	P-value
SIS total	Male	14.51 (5.76)	t(702.13) = −2.71	.01
	Female	15.56 (6.90)
Resolved plan	Male	8.05 (3.03)	t(711.06) = −1.45	.15
	Female	8.35 (3.67)
Suicidal desire	Male	6.46 (3.11)	t(681.06) = −3.64	.00
	Female	7.21 (3.62)
SIS total	Senior high school student	15.45 (6.69)	t(1188) = 0.51	.61
	University student	15.23 (6.57)
Resolved plan	Senior high school student	7.94 (3.58)	t(1188) = −2.23	.03
	University student	8.44 (3.47)
Suicidal desire	Senior high school student	7.50 (3.53)	t(1188) = 3.23	.00
	University student	6.79 (3.44)

Reliability

The results showed that SIS has good and excellent internal consistency, namely Cronbach’s Alpha = .92, .87, and .86 for the 10 items, resolved plan subscale, and suicide desire subscale, respectively. The correlation between each item score and the total item ranged between .59 and .82 for the 10 items, .59 and .76 for the resolved plan subscale, as well as .67 and .76 for the suicide desire subscale. The test-retest reliability was analyzed based on the retest data that has been collected approximately 1 to 2 weeks after the first measurement. Based on the test-retest reliability, retest data were collected from 70% of the participants, and the coefficient was .84. The test-retest reliability coefficient was calculated using Pearson product-moment correlation between the first and the second time measure, where a correlation of .1 to .3 is considered weak, .4 to .6 is moderate, .7 to .9 is strong, and 1.0 is perfect. Therefore, the test-retest reliability coefficient in this study was considered as strong.

Content Validity

The content validity of the SIS was established after evaluation by an expert panel, which consists of 2 psychiatrists, 2 psychologists, and 2 language experts, revealing a high degree of relevance and representative of the SIS. This was indicated by the high item and scale content validity index (I-CVI = 1.00, S-CVI = 1.00). In the focus group discussion, the participants remarked that the SIS was easy to comprehend and had satisfactory face validity. The pilot test of the SIS also showed that it was acceptable in terms of administration time and easy to understand. Around 30 participants joined the pilot test.

Convergent Validity

The correlation coefficient between the score of SIS and PHQ was moderate and significant (r = .59, P = .00), which is the same as SIS and BDI (r = .64, P = .00). The relationship between the score of SIS and CDI was strong and significant (r = .71, P = .00). This result indicates good concurrent validity of SIS.

Factorial Validity

A total of 2 models were tested using confirmatory factor analysis (CFA), namely (i) Model 1: a 1-factor model consisting of all items of SIS; (ii) Model 2: a 2-factor model based on the study of Luxton (2011), which comprised of resolved plan subscale (RP, item number 1, 2, 3, 4, 9, 10) and suicidal desire subscale (SD, item number 5, 6, 7, 8). From the result of the goodness of fit index in Table 5, it was discovered that the 2 CFA models are a reasonably good fit to the data, where most indices of Model 1 are better than model 2 (RMSEA and SRMR). Figure 1 shows that the loading factors of Model 1 are better than Model 2, which indicated that the scores are higher. The loading factor of model 1 and model 2 were ranging from 0.74 to 0.92 and from 0.64 to 0.88, respectively. The factor of resolved plan and suicidal desire was significantly correlated (r = .95).

Table 5.

The Goodness of Fit Indices.

Model	Chi-square, degree of freedom and P value	AGFI	CFI	NNFI	RMSEA	SRMR
Model 1	${\chi }_{\left(22\right)}^2=153.31,p=0.00$	0.94	0.99	0.98	0.07	0.02
Model 2	${\chi }_{\left(29\right)}^2=234.52,p=0.00$	0.93	0.97	0.96	0.08	0.03

Abbreviations: AGFI, adjusted goodness-of-fit index; CFI, comparative fit index; NNFI, the non-normed fit index; RMSEA, root mean squared error of approximation; SRMR, standardized root mean square residual.

Figure 1.

The 2-factor models of the Suicidal Ideation Scale. All loading factors and the correlation coefficient between subscales in model 2 are significant at 5%. Abbreviations: RP, subscale of resolved plan; SD, suicidal desire. SIS_1, SIS_2, . . ., SIS_10 = item1, item2, . . ., item10 of suicidal ideation scale.

Discussion

This study reported the adaptation and validation process of the Suicidal Ideation Scale (SIS) in the Indonesian language among senior high school and university students. The results showed that SIS has good to excellent internal consistency and test-retest reliability. Based on the validity indicators, it has satisfactory content and convergent validity, and further support the one-factor and 2-factor model for factorial validity.

The results also showed that the prevalence of suicidal ideation among senior high school and university students is 15%. This number is similar with previous studies, with 7.9% among adolescents in Malaysia, 15.6% in the United States, and 16.4% during the COVID-19 pandemic in China.^33-35 However, the values obtained in this study were lower than that of Rudd,⁹ who reported that the prevalence of suicidal ideation among college students was 43.7%.

Based on gender differences, it was discovered that female students have a higher suicidal desire than their male counterparts. This is in line with previous studies, where the rates of lifetime suicidal ideation, plans, and attempts are higher in females than males,^36,37 including during the COVID-19 pandemic.³⁸ According to Miranda-Mendibazal et al,³⁶ females are more prone to anxiety and mood disorders, these factors were associated with increased suicidal thoughts and behavior.

There was no difference in prevalence among the 2 age groups, namely high school and university students. This is in line with the reports by Laghi et al³⁹ and Gonçalves et al,⁴⁰ where the prevalence of suicidal ideation was 7.6% and 7.8% among high school and university students, respectively. When the factors were observed, it was discovered that high school students tend to score higher on the suicidal desire, but lower on the resolved plan subscale. This indicates that high school students are likely to have a suicidal desire, but without further planning or preparations. Meanwhile, university students tend to plan suicide more seriously instead of ideas or desires. According to Pompili et al,⁴¹ people who attained higher educational levels are more prone to suicide due to failures faced, public shame, and high premorbid functioning. Therefore, university students tend to have more capability to commit suicide. Stoliker et al⁴² also explained that those who are unable to cope with life’s changes have more potential to enact suicidal behavior, especially when they have an acquired capability for suicide such as a lowered fear of death in older people and higher pain tolerance.

The Indonesian version of the SIS is a reliable instrument that shows high internal consistency and stability over time (test-retest). The internal consistency is almost similar with the finding of Ibrahim et al⁴³ Moreover, the results based on both internal consistency and corrected total item are at least similar or higher compared to the investigations of Rudd and Luxton et al.^9,10 This study also added further evidence of the satisfactory test-retest reliability of SIS. Further comparison of the results in this study with previous studies could be seen in Supplemental Table 1.

Based on content validity analysis, all experts involved in the translation process agreed that the result was equivalence between the source and target version. The qualitative process was similar to the study of Han et al.¹¹ However, this study gives more information on the quantitative perspective by measuring CVI, which showed excellent index validity.²⁸

The convergent validity of the Indonesian version of the SIS is shown by a moderate to high level of correlation with several depression scales (PHQ, BDI, and CDI), which is consistent with previous studies.^9,44 They used Ultra-Short Suicidal Ideation Scale (USSIS) and Geriatric Depression Scale–Short Form (GDS-SF) in their studies as the criterion.

The Indonesian version of the SIS had good evidence of factorial validity. This study highlighted that even though both models are considered to be a good fit for the data, Model 1 has a higher loading factor of 0.74 to 0.92 compared to Model 2, with a value of 0.62 to 0.88. Therefore, the one-factor model is more preferred than the 2-factor model to identify suicidal ideation. Even though this study was performed in non-clinical settings, the result is still consistent with previous studies in clinical populations.^10,11 Therefore, showing that the 2-factor model of SIS is also suitable in both clinical and non-clinical settings.

Conclusion

The Indonesian version of the Suicidal Ideation Scale (SIS) showed excellent internal consistency using Alpha Cronbach, and the test-retest reliability coefficient in this study was considered excellent. In addition, the content, convergent, and factorial validity were proven acceptable in non-clinical settings.

The SIS is proven to be a valid and reliable tool, which can be used among adolescents and young adults to assess suicidal ideation in non-clinical populations. It could also fulfill the need for development of suicide prevention strategies and policies in Indonesia that targets non-clinical populations such as groups and institutional levels.

The main limitation is that data were collected during the COVID-19 pandemic, which can influence the suicidal ideation of adolescents. Therefore, further investigation on the impact of the COVID-19 pandemic on suicidal ideation among adolescents in Indonesia is recommended. Another drawback of this study is the stigmatization of suicidal ideation in the country, which is even considered taboo, leading to under-reported by participants.⁴⁵