Abstract
Introduction:
This study aimed to demonstrate the validity of the Ask Suicide-Screening Questions (ASQ) in a clinical sample consisting of adolescents admitted to child and adolescent psychiatry (CAP), and then to confirm its validation in those presenting to the pediatric emergency department (PED), which was the main target group for the study.
Method:
This cross-sectional study evaluated the compatibility of the ASQ with the suicide probability scale, which is a standardized measure, to identify cases with suicide risk in 248 adolescents aged 10–18 years. To demonstrate the clinical validity of the scale, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), negative likelihood ratio (NLR), Kappa, and area under the curve (AUC) performance metrics and 95% confidence interval (CI) values were calculated.
Results:
Positive screening rate, sensitivity, specificity, PPV and NPV for the CAP patients were calculated as 31.8%, 100% (95% CI: 100.0–100.0), 70.9% (95% CI: 63.4–78.4), 12.8% (95% CI: 3.2–22.3) and 100% (95% CI: 100.0–100.0), respectively. The PLR and AUC were calculated as 3.4% (95% CI: 2.7–4.5) and 0.855 (95% CI: 0.817–0.892), respectively. Positive screening rate, sensitivity, specificity, PPV and NPV for the PED patients were calculated as 28%, 100% (95% CI: 100.0–100.0), 75.3% (95% CI: 66.3–84.2), 21.4% (95% CI: 6.2–36.6) and 100% (95% CI: 100.0–100.0), respectively. The PLR, Kappa and AUC were 4.05% (95% CI: 2.82–5.81), 0.278 and 0.876 (95% CI: 0.832–0.921), respectively.
Conclusion:
This study showed the first evidence that Turkish adaptation of the ASQ is a valid screening tool for identifying those at risk of suicide among adolescents who applied to the CAP and PED.
Keywords: Adolescent, sensitivity and specificity, suicide
INTRODUCTION
Suicide, a public health concern, is one of the most common causes of death among adolescents (1). Eighty percent of the young people who lost their lives as a result of suicide consult a physician with somatic symptoms in the last three months before their death, and do not talk about suicidal thoughts unless asked (2). In our country, there is no routine screening for suicide risk in this age group, except for adolescent mental health services. Considering that approximately one-third of hospital admissions in Turkey are to emergency services, for most adolescents, the pediatric emergency service is the only place they encounter healthcare providers. Providing suicide screening during their emergency service visit is an important opportunity to identify adolescents at risk (3–6).
In our country, valid and reliable assessment tools can be used to evaluate suicide risk in adolescents (7,8). However, the primary screening use of these scales in pediatric emergency services is limited due to the long application time and the complexity of scoring. Instead, there is a need for a short suicide screening tool that can be applied quickly and easily, does not require practitioner training, and can be used with children who have mental and/or physical problems. For this purpose, the Ask Suicide-Screening Questions (ASQ) made up of 5 questions that can be asked to both outpatient and inpatient diagnosis/treatment units has been developed (9–11). With the recommendation of the Physically Ill Child Committee of the American Academy of Child and Adolescent Psychiatry, the ASQ is currently routinely applied during triage by nurses in the pediatric emergency services of some hospitals (12,13).
In the United States, the ASQ scale was applied for three years to 91,850 young people aged between 10 and 17 years who applied to the health system, to screen for suicide risk, and the data of 2,387 inpatients, 79,616 outpatients, and 9,577 patients who applied to the emergency department were analyzed retrospectively (14). Positive suicidal risk screening in outpatients was found to be 2.2%, 8.5% in emergency service patients, 2.1% in inpatient service patients, and 2.9% in all patients. The high rate of positive screening emphasizes the importance of screening every child who enters the health system, especially the pediatric emergency service, with the ASQ scale, and draws attention to the need to develop similar health policies in our country.
Highlights
Ask Suicide-Screening Questions (ASQ) can be applied quickly and easily.
ASQ can be used in adolescents aged 10–18 years with mental or physical problems.
The Turkish version of the ASQ is a valid tool for determining the risk of suicide.
The clinical use of the ASQ should be expanded in the pediatric population.
This study aimed to evaluate the psychometric properties of the Turkish translation of the ASQ scale in patients who applied to the child and adolescent mental health and diseases outpatient clinic and the pediatric emergency service. It aimed to demonstrate the validity of the Turkish version of the ASQ in a clinical sample consisting of adolescents and to compare all cases screened as positive and negative according to the scale in terms of sociodemographic and clinical characteristics.
METHODS
Participants
This cross-sectional study was conducted between 01.07.2020 and 01.05.2022 at Ankara University Faculty of Medicine Child and Adolescent Psychiatry Outpatient Clinic and Pediatric Emergency Service. The sample consisted of children between the ages of 10 and 18 who applied to these units as outpatients to receive health services, were able to answer the questions medically, and voluntarily agreed to participate in the study. In valid and reliable studies, it is recommended that the sample size should be at least 5 to 10 times the number of items in the relevant scale (18). Accordingly, after giving detailed information about the study, 248 adolescents whose parents and themselves gave informed consent were included in this study.
Procedure
In order to conduct the study, approval was obtained from the Clinical Research Ethics Committee of Ankara University Faculty of Medicine (Decision No: 16-328-20/Date: 18.06.2020). Permission was obtained by contacting Dr. Horowitz et al. to adapt the scale to Turkish. The Turkish translation of the scale was made by a specialist working in the field of child and adolescent psychiatry who is fluent in English and experienced in similar studies. The translated scale was then translated back to English and the translated and back-translated versions were carefully compared with the original English version by Dr. Horowitz et al. Finally, necessary corrections were made and the Turkish version of the ASQ scale was given its final form. Adolescents who agreed to participate in the study were asked to fill in the suicide probability scale, a standardized measurement tool, to obtain data on clinical validity by evaluating the compatibility with the Turkish version of the scale.
In addition, sociodemographic information form, strength and difficulty questionnaire adolescent form, and children’s depression inventory were filled in by the participants in order to compare all cases screened as positive and negative according to the scale in terms of sociodemographic and clinical characteristics. The Turkish version of the ASQ was administered to the adolescents separately from their parents, if possible, with their parents’ permission. Adolescents who screened positive on ASQ underwent a brief suicide safety assessment, and those requiring further intervention based on the degree of risk were referred for further evaluation.
Measures
The Sociodemographic Information Form
This form, prepared by the researchers, questions sociodemographic characteristics of adolescents such as age, sex, educational status, family structure, number of siblings, family income, their parents’ age, and educational status. The socioeconomic status of the family, according to the Hollingshead-Redich scale, was classified as “wealthy, educated family”, “professional or high administrator, college graduate parents”, “small business owner, white collar or skilled worker, high school graduate parents”, “semi-skilled worker, parents with less than high school education”, “semi-skilled worker, uneducated or primary school graduate parents” and “unknown”.
Ask Suicide-Screening Questions
Ask Suicide-Screening Questions (ASQ) developed by Dr. Horowitz et al. consists of five screening questions that take healthcare professionals approximately 20 seconds to ask adolescents aged between 10 and 24. The scale was first validated on 524 patients aged from 10 to 21 years who applied to the pediatric emergency service with mental or physical complaints. While investigating the validation of the scale, the sensitivity was determined as 96.9% and specificity as 87.6%, and negative predictive values were found to be 99.7% for the patients presenting with physical complaints and 96.9% for those presenting with mental complaints (9). Then, it was confirmed that the scale showed strong psychometric properties in young people who applied to pediatric outpatient or inpatient diagnosis and treatment units (10,11). The scale is free to use and has translations in Arabic, Chinese, French, Dutch, Hebrew, Italian, Japanese, Korean, Portuguese, Russian, and Spanish (https://www.nimh.nih.gov/asq). It is recommended to apply the scale to the patient alone, if possible, by asking the parents their permission. A clinical algorithm based on the ASQ which directs the healthcare team to appropriate intervention according to the patient’s suicide degree has been developed by the Physically Ill Child Committee of the American Academy of Child and Adolescent Psychiatry (12). In this way, the ASQ plays an important role in preventing suicide by providing early detection of high suicide risk and further evaluation of young people in pediatric emergency, pediatric health, and diseases outpatient or inpatient diagnosis and treatment units (13).
As seen in Figure 1, giving a “yes” answer to at least one of the first four questions of the ASQ or refusing to answer is defined as a positive screening, and the patient is considered be at possible suicide risk. Answering “no” to all of the first four questions is defined as a negative screening and no additional intervention is required. The decision made as a result of clinical evaluation can always invalidate negative screening. In case of possible suicide risk, the fifth question is asked to clarify the risk. Answering “no” to this question is considered non-acute positive screening and a short standardized safety assessment lasting approximately 10 minutes is performed by clinicians trained in suicide risk assessment. A patient’s suicide risk is graded through the brief suicide safety assessment that is a semi-structured clinical interview and can be classified as low-risk, high-risk, and imminent high-risk based on clinical opinion. (https://www.nimh.nih.gov/asq">https://www.nimh.nih.gov/asq">https://www.nimh.nih.gov/asq).
Figure 1.
Clinical algorithm based on the ASQ recommended by the Physically Ill Child Committee of the American Academy of Child and Adolescent Psychiatry.
There is no need for a comprehensive suicide safety assessment for a low-risk patient, simple safety precautions are explained to the patients and their family, and the patient is discharged by referring to the child and adolescent mental health and disease clinic for evaluation at a suitable time. In the case of a high-risk patient, a comprehensive suicide safety assessment is performed by clinicians trained in the field of child and adolescent mental health and diseases, and patients who require hospitalization are transferred to the child and adolescent mental health and diseases inpatient service. Detailed safety precautions are explained to patients who do not require hospitalization and their families, and a close follow-up appointment is scheduled. The patient is referred to the child and adolescent mental health and diseases outpatient clinic for an evaluation in the near future (preferably within 72 hours) and discharged.
The imminent high-risk situation is approached in the same way as with patients who answered the fifth question with a “yes”. Both conditions are referred to as acute positive screening and are considered to be at a high risk of suicide. A comprehensive suicide safety assessment is performed urgently by clinicians trained in the field of child and adolescent mental health and diseases. Safety measures such as keeping the patient under observation and removing dangerous objects and substances are put into practice immediately, and the severity of the situation is reported to the pediatric emergency service medical team members and parents.
A comprehensive suicide safety assessment is an unstructured clinical interview; the patient’s risk factors for suicidal behavior are discussed, and a differential diagnosis and treatment plan is made in cooperation with the patient and their family, and the patient is transferred to the pediatric and adolescent mental health and diseases inpatient service (12).
Suicide Probability Scale
The Suicide Probability Scale (SPS) was developed by Cull and Gill to identify adolescents and adults at risk of committing suicide (16). The predictive validity study of the scale showed that the scores obtained from SPS correctly predict suicide attempts and self-harming behaviors in adolescents (17). SPS consists of a total of four subscales; hopelessness, negative self-evaluation, hostility, and suicidal ideation, and 36 items. Each subject is scored between 1–4 on a four-point Likert-type scale and can be answered with the options of “never or rarely”, “sometimes”, “often” and “often or always”. The total score obtained from the scale varies between 36 and 144. The Turkish version of the scale is a valid and reliable tool that can be used in both clinical and population-based surveys between the ages of 14–76 (7). The Cronbach’s alpha coefficient of the Turkish version was found to be 0.89 for the whole sample, and the cut-off point was determined as 110.
Strengths and Difficulties Questionnaire self-report version
The Strengths and Difficulties Questionnaire Adolescent Form (SDQ-AF) was developed to screen for emotional and behavioral problems in adolescents aged between 11 and 16 years, based on self-reports (18). This questionnaire also has a parent and teacher version for children aged between 4 and 16. It consists of a total of 25 items and five subscales; behavioral problems, attention deficit and hyperactivity, emotional problems, peer problems, and social behaviors. The total difficulty score of the scale is calculated with the sum of the four subscale scores, excluding the social behaviors subscale. The Turkish adaptation of the scale was confirmed by Güvenir et al. (19). The Cronbach’s alpha coefficient of the Turkish version was 0.73.
Children’s Depression Inventory
Kovacs developed the Children’s Depression Inventory (CDI) to evaluate depressive symptoms in children aged between 6 and 17 years (20). The scale is a 27-item single-factor self-report tool. Each item gets 0, 1, or 2 points depending on the severity of the symptom. The maximum score that can be obtained from the scale is 54. The cut-off score of the scale is recommended as 19. The Turkish version of the scale was found to have good psychometric properties, and the Cronbach’s alpha coefficient was calculated as 0.87 to evaluate its internal consistency (21).
Statistical Analysis
In this study, data entry and data analysis were performed using the IBM Statistical Package for the Social Sciences (SPSS) version 23. Descriptive statistics were presented as mean, standard deviation, median, 1st and 3rd quartile values for numerical variables, and as numbers and percentages for categorical variables. Comparisons between categorical variables were made using the Pearson chi-square test when the chi-square assumption was met. In the absence of the chi-square assumption, the Fisher exact chi-square test for 2x2 dimensional tables and the exact chi-square test for table structures other than 2x2 were used. The Kolmogorov-Smirnov goodness-of-fit test was used to determine whether the data of numerical variables were in accordance with the normal distribution. The homogeneity of the variances was examined by Levene’s test. When the assumptions of normality and homogeneity of variances were met, comparisons between two independent groups were made using the independent samples t-test, and if not, the Mann-Whitney U test was used. In previous studies, it has been shown in adolescents that SPS scores are a moderate/high-level sensitive marker for subsequent suicide attempts (17,22). Therefore, the ASQ was used as a reference (gold standard) test and compared with the ASQ, and the data on the validity of the ASQ were obtained by evaluating its compatibility with the ASQ. For this, correct classification rate, sensitivity, specificity, positive predictive value, negative predictive value , positive likelihood ratio, negative likelihood ratio, Kappa, and area under the curve (AUC) performance metrics and 95% confidence interval (CI) (95% CI) values were calculated using the “reportROC” library in the R programming language. The statistical significance level was taken as p<0.05.
RESULTS
Descriptive statistics regarding the children participating in the study and their families can be seen in Table 1. When the application centers of the participants were examined; 59.7% (n=148) of the sample consisted of those who applied to the child and adolescent mental health and diseases outpatient clinic, and 40.3% (n=100) to the pediatric emergency service. According to the statistics given in Table 1, there was no statistically significant difference found between the patients who applied to the child and adolescent mental health and diseases outpatient clinic and those who applied to the pediatric emergency service in terms of variables such as sex, age, family structure, number of siblings, and maternal and paternal age. The median values of mother and father education periods of the patients who applied to the child and adolescent mental health and diseases outpatient clinic were found to be statistically significantly higher than those of the patients who applied to the pediatric emergency service (Table 1).
Table 1.
Sociodemographic characteristics of patients who applied to the child and adolescent mental health and diseases outpatient clinic and pediatric emergency department
| CAP | PED | Total | ||||||
|---|---|---|---|---|---|---|---|---|
| n | %d | n | %d | n | %d | p | ||
| Sex (n=248) | ||||||||
| Male | 77 | 52.0 | 41 | 41.0 | 118 | 47.6 | 0.088a | |
| Female | 71 | 48.0 | 59 | 59.0 | 130 | 52.4 | ||
| Age (years) (n=234) | ||||||||
| Mean ± SD | 14.6±2.0 | 14.8±1.9 | 14.7±1.9 | 0.447b | ||||
| Median (1. Quart. – 3. Quart.) | 15.0 (13.0–16.0) | 15.0 (14.0–16.0) | 15.0 (13.0–16.0) | |||||
| Family structure (n=206) | ||||||||
| Nuclear family | 77 | 72.6 | 82 | 82.0 | 159 | 77.2 | 0.091c | |
| Extended family | 2 | 1.9 | 4 | 4.0 | 6 | 2.9 | ||
| Broken family | 27 | 25.5 | 14 | 14.0 | 41 | 19.9 | ||
| Number of siblings (n=206) | ||||||||
| None | 19 | 17.9 | 10 | 10.0 | 29 | 14.1 | 0.056a | |
| 1 sibling | 53 | 50.0 | 43 | 43.0 | 96 | 46.6 | ||
| Least 2 siblings | 34 | 32.1 | 47 | 47.0 | 81 | 39.3 | ||
| Grade (n=204) | ||||||||
| Middle school (5–8. Grade) | 44 | 41.9 | 37 | 37.4 | 81 | 39.7 | 0.509a | |
| High school (9–12. Grade) | 61 | 58.1 | 62 | 62.6 | 123 | 60.3 | ||
| Father’s age (years) (n=202) | ||||||||
| Mean ± SD | 46.7±7.0 | 46.3±6.3 | 46.5±6.7 | 0.871b | ||||
| Median (1. Quart. – 3. Quart.) | 46.0 (41.0–50.0) | 46.0 (42.0–51.0) | 46.0 (42.0–50.0) | |||||
| Mother’s age (years) (n=203) | ||||||||
| Mean ± SD | 42.2±7.4 | 42.0±5.8 | 42.1±6.7 | 0.578b | ||||
| Median (1. Quart. – 3. Quart.) | 42.0 (38.0–47.0) | 41.0 (38.0–46.5) | 41.0 (38.0–47.0) | |||||
| Father’s education (years) (n=202) | ||||||||
| Mean ± SD | 10.5±3.7 | 9.6±3.4 | 10.1±3.6 | 0.041b | ||||
| Median (1. Quart. – 3. Quart.) | 12.0 (8.0–12.0) | 8.0 (8.0–12.0) | 11.0 (8.0–12.0) | |||||
| Mother’s education (years) (n=203) | ||||||||
| Mean ± SD | 10.2±4.0 | 8.9±3.5 | 9.6±3.8 | 0.006b | ||||
| Median (1. Quart. – 3. Quart.) | 12.0 (8.0–12.0) | 8.0 (5.0–12.0) | 10.0 (5.0–12.0) | |||||
| Socioeconomic status (according to Hollingshead-Redich Scale) (n=205) | ||||||||
| Wealthy, educated family | –- | –- | 1 | 1.0 | 1 | 0.5 | 0.032c | |
| Professional or high administrator, college graduate parents | 21 | 20.0 | 12 | 12.0 | 33 | 16.1 | ||
| Small business owner, white collar or skilled worker, high school graduate parents | 50 | 47.6 | 40 | 40.0 | 90 | 43.9 | ||
| Semi-skilled worker, parents with less than high school education | 17 | 16.2 | 32 | 32.0 | 49 | 23.9 | ||
| Semi-skilled worker, uneducated or primary school graduate parents | 15 | 14.3 | 15 | 15.0 | 30 | 14.6 | ||
| Unknown | 2 | 1.9 | –- | –- | 2 | 1.0 | ||
a: Pearson chi-square test; b: Mann-Whitney U test; c: Exact chi-square test; d: Column percentage 1. Quart. – 3. Quart.: 1st quartile and 3rd quartile values; CAP: Child and adolescent psychiatry; Mean ± SD: Mean ± standard deviation; PED: Pediatric emergency department
Statistics regarding the clinical characteristics of the patients who applied to the child and adolescent mental health and diseases outpatient clinic and pediatric emergency service can be seen in Table 2. 29.2% (n=31) of the patients who applied to the child and adolescent mental health and diseases outpatient clinic had a history of physical illness, and statistically, this rate was found to be significantly higher than those who applied to the pediatric emergency service. The median value of the behavioral problems subscale score of the SDQ adolescent form filled out by the patients who applied to the child and adolescent mental health and diseases outpatient clinic was found to be statistically significantly higher than that of the patients who applied to the pediatric emergency service. As seen in Table 2, there was no statistically significant difference between referral centers in terms of other clinical characteristics. Those who answered “yes” to suicide screening questions from patients who applied to the child and adolescent mental health and diseases outpatient clinic and pediatric emergency services are presented in Table 3. There is no statistically significant difference between referral centers in terms of “yes” answers to suicide screening questions.
Table 2.
Clinical characteristics of patients who applied to the child and adolescent mental health and diseases outpatient clinic and pediatric emergency department
| n=206 | CAP | PED | Total | |||||
|---|---|---|---|---|---|---|---|---|
| n | %d | n | %d | n | %d | p | ||
| History of physical illness | ||||||||
| Yes | 31 | 29.2 | 16 | 16.0 | 47 | 22.8 | 0.024a | |
| No | 75 | 70.8 | 84 | 84.0 | 159 | 77.2 | ||
| Past suicide attempt | ||||||||
| Yes | 12 | 11.3 | 12 | 12.0 | 24 | 11.7 | 0.879a | |
| No | 94 | 88.7 | 88 | 88.0 | 182 | 88.3 | ||
| Family history of suicide | ||||||||
| Yes | 5 | 4.7 | 7 | 7.0 | 12 | 5.8 | 0.484a | |
| No | 101 | 95.3 | 93 | 93.0 | 194 | 94.2 | ||
| Physical illness in the family | ||||||||
| Yes | 41 | 38.7 | 28 | 28.0 | 69 | 33.5 | 0.105a | |
| No | 65 | 61.3 | 72 | 72.0 | 137 | 66.5 | ||
| CDI | ||||||||
| Mean ± SD | 15.9±9.8 | 15.1±9.7 | 15.6±9.8 | 0.469b | ||||
| Median (1. Quart. – 3. Quart.) | 15.0 (8.0–22.0) | 13.0 (7.0–21.0) | 14.0 (8.0–21.0) | |||||
| SDQ | ||||||||
| Emotional problems | Mean ± SD | 3.7±2.6 | 4.2±2.5 | 3.9±2.6 | 0.179b | |||
| Median (1. Quart. – 3. Quart.) | 3.0 (2.0–6.0) | 4.0 (2.0–6.0) | 4.0 (2.0–6.0) | |||||
| Behavioral problems | Mean ± SD | 2.8±2.0 | 2.3±2.0 | 2.6±2.0 | 0.024b | |||
| Median (1. Quart. – 3. Quart.) | 3.0 (1.0–4.0) | 2.0 (1.0–3.0) | 2.0 (1.0–4.0) | |||||
| Attention deficit/hyperactivity | Mean ± SD | 4.9±2.3 | 5.4±2.4 | 5.0±2.3 | 0.076b | |||
| Median (1. Quart. – 3. Quart.) | 5.0 (3.0–6.0) | 6.0 (4.0–7.0) | 5.0 (3.0–7.0) | |||||
| Peer problems | Mean ± SD | 3.4±1.8 | 3.5±2.1 | 3.4±1.9 | 0.689b | |||
| Median (1. Quart. – 3. Quart.) | 3.0 (2.0–5.0) | 4.0 (2.0–5.0) | 3.0 (2.0–5.0) | |||||
| Social behaviors | Mean ± SD | 7.4±2.4 | 7.4±2.1 | 7.4±2.3 | 0.697b | |||
| Median (1. Quart. – 3. Quart.) | 8.0 (6.0–9.0) | 8.0 (6.0–9.0) | 8.0 (6.0–9.0) | |||||
| Total difficulty score | Mean ± SD | 14.8±6.7 | 15.4±6.8 | 15.0±6.7 | 0.546c | |||
| Median (1. Quart. – 3. Quart.) | 14.0 (10.0–20.0) | 15.0 (9.5–20.0) | 15.0 (10.0–20.0) | |||||
a: Pearson chi-square test; b: Mann-Whitney U test; c: Independent groups t-test; d: Column percentage 1. Quart. – 3. Quart.: 1st quartile and 3rd quartile values; CAP: Child and adolescent psychiatry; CDI: Depression scale for children; Mean ± SD: Mean ± standard deviation; PED: Pediatric emergency department; SDQ: Strengths and difficulties questionnaire
Table 3.
Screening of patients who applied to the child and adolescent mental health and diseases outpatient clinic and pediatric emergency department with the Ask Suicide-Screening Questions
| n=248 | CAP | PED | Total | ||||
|---|---|---|---|---|---|---|---|
| Yes | n | %b | n | %b | n | %b | p |
| Ask Suicide-Screening Questions | |||||||
| 1. In the past few weeks, have you wished you were dead? | 30 | 20.3 | 24 | 24.0 | 54 | 21.8 | 0.485a |
| 2. In the past few weeks, have you felt that you or your family would be better off if you were dead? | 31 | 20.9 | 17 | 17.0 | 48 | 19.4 | 0.440a |
| 3. In the past week, have you been having thoughts about killing yourself? | 23 | 15.5 | 15 | 15.0 | 38 | 15.3 | 0.908a |
| 4. Have you ever tried to kill yourself? | 19 | 12.8 | 12 | 12.0 | 31 | 12.5 | 0.845a |
| 5. Are you having thoughts of killing yourself right now? | 14 | 9.5 | 7 | 7.0 | 21 | 8.5 | 0.495a |
a: Pearson chi-square test; b: Column percentage CAP: Child and adolescent psychiatry; PED: Pediatric emergency department
Sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, Kappa, and area under the curve performance metrics for both sub-samples are presented in Table 4. The positive screening rate with the ASQ is 31.8% (n=47) and the acute positive screening rate is 9.5% (n=14) in patients who applied to the child and adolescent mental health and diseases outpatient clinic. 4.1% (n=6) of those who applied to this outpatient clinic had an SPS score above the cut-off point of the scale. Of the patients admitted to the pediatric emergency department, 28% (n=28) were evaluated as a positive screening for ASQ, and 7% (n=7) as acute positive screening (See Table 4). 6.3% (n=6) of those who applied to the pediatric emergency service had an SPS score above the cut-off point of the scale.
Table 4.
Data on the validity of the Ask Suicide-Screening Questions in patients who applied to the child and adolescent mental health and diseases outpatient clinic and pediatric emergency department
| CAP | PED | ||
|---|---|---|---|
| n | 148 | 100 | |
| SPS | Positive, n (%) | 6 (4.1) | 6 (6.3) |
| ASQ | Positive, n (%) | 47 (31.8) | 28 (28.0) |
| ASQ-SPS compliance | True positive, n | 6 | 6 |
| True negative, n | 100 | 67 | |
| False positive, n | 41 | 22 | |
| False negative, n | 0 | 0 | |
| Correct classification rate | % (95% CI) | 72.1 (71.8–72.4) | 76.8 (76.5–77.2) |
| Sensitivity | % (95% CI) | 100.0 (100.0–100.0) | 100.0 (100.0–100.0) |
| Specificity | % (95% CI) | 70.9 (63.4–78.4) | 75.3 (66.3–84.2) |
| Positive predictive value | % (95% CI) | 12.8 (3.2–22.3) | 21.4 (6.2–36.6) |
| Negative predictive value | % (95% CI) | 100.0 (100.0–100.0) | 100.0 (100.0–100.0) |
| Positive likelihood ratio | % (95% CI) | 3.4 (2.7–4.5) | 4.05 (2.82–5.81) |
| Negative likelihood ratio | % (95% CI) | 0.0 (0.0-NaN) | 0.0 (0.0-NaN) |
| Kappa | 0.166 | 0.278 | |
| Area under the curve | AUC (95% CI) | 0.855 (0.817–0.892) | 0.876 (0.832–0.921) |
95% CI: 95% Confidence interval; ASQ: Ask Suicide-Screening Questions; AUC: Area under the curve; CAP:Child and adolescent psychiatry; NaN: Not a number; PED: Pediatric emergency department; SPS: Suicide probability scale
The sociodemographic characteristics of the cases that screened positive and negative according to the scale are presented in Table 5. According to the statistics in Table 5, a statistically significant difference was found between the cases that screened positive and negative with ASQ in terms of sex and grade distribution. No significant difference was found in terms of other sociodemographic characteristics (See Table 5). The clinical features of the cases that screened positive and negative according to the scale are given in Table 6.38.1% (n=24) of the positively screened cases had a history of suicide attempts, and 15.9% (n=10) had a family history of suicide, which was statistically significantly higher than that of the cases screened negatively (See Table 6). When the self-report-based CDI and SDQ scores in Table 6 are examined, the median scores of the CDI score, SDQ emotional problems, behavioral problems, attention deficit, and hyperactivity, and peer problems subscale scores and the mean score of the SDQ total difficulty score of those who were positively screened were found to be statistically significantly higher than those with negative scans.
Table 5.
Sociodemographic characteristics of the cases screened as positive and negative according to the Ask Suicide-Screening Questions
| ASQ | ||||||
|---|---|---|---|---|---|---|
| Negative | Positive | |||||
| n | %d | n | %d | p | ||
| Sex (n=248) | ||||||
| Male | 97 | 56.1 | 21 | 28.0 | <0.001a | |
| Female | 76 | 43.9 | 54 | 72.0 | ||
| Age (years) (n=234) | ||||||
| Mean ± SD | 14.6±2.0 | 15.0±1.8 | 0.07b | |||
| Median (1. Quart. – 3. Quart.) | 14.0 (13.0–16.0) | 15.0 (14.0–16.0) | ||||
| Family structure (n=206) | ||||||
| Nuclear family | 115 | 80.4 | 44 | 69.8 | 0.087c | |
| Extended family | 5 | 3.5 | 1 | 1.6 | ||
| Broken family | 23 | 16.1 | 18 | 28.6 | ||
| Number of siblings (n=206) | ||||||
| None | 17 | 11.9 | 12 | 19.0 | 0.341a | |
| 1 sibling | 70 | 49.0 | 26 | 41.3 | ||
| Least 2 siblings | 56 | 39.2 | 25 | 39.7 | ||
| Grade (n=204) | ||||||
| Middle school (5–8. grade) | 66 | 46.5 | 15 | 24.2 | 0.003a | |
| High school (9–12. grade) | 76 | 53.5 | 47 | 75.8 | ||
| Father’s age (years) (n=202) | ||||||
| Mean ± SD | 46.1±6.6 | 47.4±6.7 | 0.164b | |||
| Median (1. Quart. – 3. Quart.) | 45.0 (41.0–50.0) | 48.0 (42.0–51.0) | ||||
| Mother’s age (years) (n=203) | ||||||
| Mean ± SD | 41.9±6.1 | 42.4±7.9 | 0.212b | |||
| Median (1. Quart. – 3. Quart.) | 40.0 (38.0–46.0) | 42.0 (38.0–47.0) | ||||
| Father’s education (years) (n=202) | ||||||
| Mean ± SD | 10.2±3.7 | 9.8±3.3 | 0.613b | |||
| Median (1. Quart. – 3. Quart.) | 11.0 (8.0–12.0) | 8.0 (8.0–12.0) | ||||
| Mother’s education (years) (n=203) | ||||||
| Mean ± SD | 9.6±3.8 | 9.5±3.8 | 0.879b | |||
| Median (1. Quart. – 3. Quart.) | 8.0 (5.0–12.0) | 12.0 (5.0–12.0) | ||||
| Socioeconomic status (according to Hollingshead-Redich Scale) (n=205) | ||||||
| Wealthy, educated family | 1 | 0.7 | –- | –- | 0.915c | |
| Professional or high administrator, college graduate parents | 24 | 16.9 | 9 | 14.3 | ||
| Small business owner, white collar or skilled worker, high school graduate parents | 60 | 42.3 | 30 | 47.6 | ||
| Semi-skilled worker, parents with less than high school education | 34 | 23.9 | 15 | 23.8 | ||
| Semi-skilled worker, uneducated or primary school graduate parents | 21 | 14.8 | 9 | 14.3 | ||
| Unknown | 2 | 1.4 | –- | –- | ||
a: Pearson chi-square test; b: Mann-Whitney U test; c: Exact chi-square test; d: Column percentage 1. Quart. – 3. Quart.: 1st quartile and 3rd quartile values; CAP: Child and adolescent psychiatry; Mean ± SD: Mean ± standard deviation; PED: Pediatric emergency department
Table 6.
Clinical characteristics of the cases screened as positive and negative according to the Ask Suicide-Screening Questions
| ASQ | ||||||
|---|---|---|---|---|---|---|
| n=206 | Negative | Positive | ||||
| n | %d | n | %d | p | ||
| History of physical ıllness | ||||||
| Yes | 30 | 21.0 | 17 | 27.0 | 0.3444a | |
| No | 113 | 79.0 | 46 | 73.0 | ||
| Past suicide attempt | ||||||
| Yes | 0 | 0.0 | 24 | 38.1 | <0.001a | |
| No | 143 | 100.0 | 39 | 61.9 | ||
| Family history of suicide | ||||||
| Yes | 2 | 1.4 | 10 | 15.9 | <0.001b | |
| No | 141 | 98.6 | 53 | 84.1 | ||
| Physical illness in the family | ||||||
| Yes | 44 | 30.8 | 25 | 39.7 | 0.341a | |
| No | 99 | 69.2 | 38 | 60.3 | ||
| CDI | ||||||
| Mean ± SD | 11.3±6.8 | 24.7±9.1 | <0.001c | |||
| Median (1. Quart. – 3. Quart.) | 10.0 (6.0–15.0) | 24.0 (18.0–31.0) | ||||
| SDQ | ||||||
| Emotional problems | Mean ± SD | 3.7±2.6 | 5.2±2.4 | <0.001c | ||
| Median (1. Quart. – 3. Quart.) | 3.0 (2.0–6.0) | 5.0 (3.0–7.0) | ||||
| Behavioral problems | Mean ± SD | 2.0±1.7 | 4.2±2.0 | <0.001c | ||
| Median (1. Quart. – 3. Quart.) | 2.0 (1.0–3.0) | 4.0 (3.0–6.0) | ||||
| Attention deficit/hyperactivity | Mean ± SD | 4.7±2.2 | 6.2±2.3 | <0.001c | ||
| Median (1. Quart. – 3. Quart.) | 5.0 (3.0–6.0) | 6.0 (5.0–8.0) | ||||
| Peer problems | Mean ± SD | 3.2±1.8 | 4.3±1.9 | <0.001c | ||
| Median (1. Quart. – 3. Quart.) | 3.0 (2.0–5.0) | 4.0 (3.0–6.0) | ||||
| Social behaviors | Mean ± SD | 7.6±2.1 | 6.4±2.4 | <0.001c | ||
| Median (1. Quart. – 3. Quart.) | 8.0 (6.0–9.0) | 6.0 (5.0–9.0) | ||||
| Total difficulty score | Mean ± SD | 13.7±6.1 | 19.9±6.0 | <0.001d | ||
| Median (1. Quart. – 3. Quart.) | 14.0 (9.0–19.0) | 20.0 (15.0–25.0) | ||||
a: Pearson chi-square test, b: Fisher exact chi-square test, c: Mann-Whitney U test, d: Independent groups t-test, e: Column percentage, ASQ: Ask Suicide-Screening Questions, CDI: Depression scale for children, SDQ: Strengths and difficulties questionnaire, Mean ± SD: Mean ± standard deviation, 1. Quart. – 3. Quart.: 1st quartile and 3rd quartile values.
DISCUSSION
In this study, the Turkish version of the ASQ is compared with a standardized measurement tool that determines the probability of suicide in young people, and it was aimed to verify separately in sub-samples of patients aged 10–18 years who applied to the child and adolescent mental health and diseases outpatient clinic and the pediatric emergency department with mainly mental or physical problems. In line with this aim, it has been shown that the scale is a valid screening tool for detecting young people at high risk of committing suicide in both sub-samples, which are generally similar in terms of sociodemographic and clinical characteristics. It has been determined that there are differences in some sociodemographic and clinical characteristics that may be important in terms of suicide risk among young people who screened positive and negative on ASQ. Thus, a safe, fast and easy suicide screening scale that does not require practitioner training and can even be used with children who have physical problems has become available in our country.
In order to demonstrate the validity of the suicide screening questions, the Turkish version of the scale was compared with the SPS, and its ability to distinguish young people at risk from others was examined. The sensitivity value, which is interpreted as the capacity of ASQ to detect a high risk of suicide, was calculated as 100% for both sub-samples. Thus, it has been shown that the Turkish version of the ASQ is a very sensitive scale, all young people at risk with ASQ are screened positively, and there is no possibility of false negative screening. Similarly, in the study in which ASQ was developed, this rate was found to be 96.9% in patients who applied to the pediatric emergency department. (9).
The specificity value of the scale, which is defined as the capacity to distinguish those who are not at risk for committing suicide, was found to be 70.9% and 75.3% for the child and adolescent mental health and diseases outpatient clinic and the pediatric emergency service, respectively. It was found to be similar to the percentage calculated for young people with physical or mental problems (65.6%–93.9%) (9). These findings indicate that the rate of false positive scans with ASQ is acceptably low.
However, one of the suicide screening questions explores past suicide attempts, and the “yes” answer to this question defines the individual as a direct positive screening regardless of the time of the suicide attempt. Therefore, positive scans may not always mean that the patient is at a high risk of suicide. Nevertheless, the use of the Turkish version of the ASQ for screening purposes in the pediatric emergency service seems appropriate due to its sensitivity and specificity values (11).
The positive and negative predictive values of the Turkish version of the scale were 12.8% and 100%, respectively, in patients who applied to the child and adolescent mental health and diseases outpatient clinic, and it was found to be 21.4% and 100%, respectively, in those who applied to the pediatric emergency service. The positive predictive value is the probability that those who screen positive with ASQ actually have a high risk of suicide. In this study, positive predictive values were lower than those in the literature (39.4%–71.3%) (9).
The high sensitivity value of the scale may have caused a large amount of false positivity, and the positive predictive value might have been calculated lower than expected. The negative predictive value is the probability that those screened negative with the ASQ actually have a low risk of suicide. High negative predictive values that were found in this study are similar to previous studies’ findings (96.9% -100%) (9,11). In this way, the clinical validity of the scale was supported by showing that none of the patients who applied to the child and adolescent mental health and diseases outpatient clinic and the pediatric emergency service were evaluated as false negative in the screenings with the Turkish version of the ASQ.
Positive screening rates with suicide screening questions were found to be 31.8% in patients who applied to the child and adolescent mental health and diseases outpatient clinic and 28% in those who applied to the pediatric emergency service. Similarly, in the study where ASQ was developed, the positive screening rate was found to be 28.2% in 524 patients aged between 10 and 21 years who applied to the pediatric emergency service primarily with mental or physical problems (9). In another study where 79 patients aged from 10 to 12 years were screened in the pediatric emergency department with ASQ, this rate was reported as 29.1% (23). In a cohort study, pediatric emergency medical records were retrospectively analyzed, and it was determined that the rate of positive screening with ASQ was 58.3% in 768 patients aged between 8 and 18 years who presented with mental problems (24). On the other hand, in another retrospective cohort study, 4,666 of 15,003 children and youth aged from 8 to 18 years were admitted to the pediatric emergency service with mental problems and 10,337 with physical problems, and positive screening rates with ASQ for all samples and sub-samples were 14%, 9%, 29% and 7.8%, respectively (3). As can be seen, positive screening rates vary depending on the characteristics of the sample and study design, such as the size of the sample, admission cause of the cases, and referral centers. In addition, this study was conducted during the COVID-19 pandemic, and the high positive screening rates detected may be related to the time period of the study. In a study investigating the changes observed in terms of suicide rate and characteristics before and after the pandemic, it was found that suicide attempts increased after COVID-19 in adolescents aged between 17 and 18, and it was more common in those with mental problems and a history of suicide attempts (25).
In this study, it was determined that those who were evaluated as positive for the ASQ were more likely to be female, have a high school education level, and have a history of suicide in their personal and family history. In addition, depression, emotional problems, behavioral problems, attention deficit and hyperactivity, peer problems, and total difficulty scores were found to be higher. Suicide occurs as a result of the interaction of many etiological factors. Male sex, age range between 15 and 19, previous suicide attempts, stressful life events, socioeconomic difficulties, certain personality traits (impulsivity, hopelessness), history of trauma and abuse, presence of psychiatric disorders, exposure to discrimination, being a refugee or immigrant, alcohol and substance use, having a chronic physical illness, family conflict, having a family history of psychiatric disorders and suicidal behavior is associated with increased suicidal behavior in adolescents (1). The fact that the majority of those who screened positive on suicide screening questions had risk factors other than sex for suicidal behavior indirectly supports the clinical validity of the scale. Similar to this study, it has been shown that adolescents who are positive with ASQ in pediatric emergency services, pediatric outpatient clinics, and inpatient diagnosis and treatment units are more likely to be female (3,10,11).
When the suicide screening questions are examined, it is seen that apart from the question which examines suicide attempts in the past, other questions of the scale evaluate the suicide risk of the youth based on suicidal ideation. In this respect, the majority of those screened positive are girls, a finding that can be explained by the relationship between the female sex and increased suicidal ideation (1,9). Detection of these risk factors that increase suicidal tendencies in adolescents who screened positive on suicide screening questions shows that the scale can accurately determine those at risk of suicide.
The results of the study should be interpreted considering some of its limitations. The characteristics of the sub-samples of the study are different from those of the general adolescent population and cannot represent all children and adolescents between the ages of 10 and 18 in Turkey. Suicide screening questions need to be validated in a large multicenter sample from different geographical regions of Turkey, including pediatric emergency services, pediatric outpatient clinics, and inpatient diagnosis and treatment units in primary, secondary, and tertiary health institutions and organizations. The small sample size may have led to selection bias, resulting in an overestimation of positive screening rates. Conducting the study during the COVID-19 pandemic may have affected the sample size, selection bias, and positive screening results with ASQ. Patients who applied to the pediatric emergency service were not categorized according to their application causes such as mental or physical problems, and the positive screening rate was determined by considering them as a whole, and this may have affected the results.
In conclusion, this study presents the first evidence for the validity of using ASQ to detect suicide risk in a clinical sample of adolescents in Turkey. First of all, sensitivity and specificity analyzes were shown in the patients who applied to the child and adolescent mental health and diseases outpatient clinic, and then they were confirmed in the patients who applied to the pediatric emergency service, which was the main target group of the study. Cases that screened positive and negative according to suicide screening questions were compared in terms of sociodemographic and clinical features, and risky features were observed in terms of suicidal behavior in positive screenings. In the future, by repeating these findings in a multicenter prospective cohort study, ASQ can become routinely used in pediatric emergency services in Turkey, and it will be possible to deal effectively with a clinical condition that is one of the leading causes of preventable death in young people.
Acknowledgements:
Thanks to Dr. Fulden Aycan who made us aware of the Ask Suicide-Screening Questions (ASQ), and Dr. Lisa Horowitz and her team who allowed us to use ASQ and contributed to the Turkish translation of the scale.
Footnotes
Ethics Committee Approval: In order to conduct the study, approval was obtained from the Clinical Research Ethics Committee of Ankara University Faculty of Medicine (Decision No: 16-328-20/Date: 18.06.2020).
Informed Consent: 248 adolescents whose parents and themselves gave informed consent were included in this study.
Peer-review: Externally peer-reviewed..
Author Contributions: Concept- SDU, MÇU, BGK; Design- SDU, BGK; Supervision- MÇU, GV, FG; Resource- SDU, BGK; Materials- EAA, ÖÇB; Data Collection and/or Processing- EAA, ÖÇB, GV, FG; Analysis and/or Interpretation- SDU, HKÜ; Literature Search- MÇU, HKÜ; Writing- SDU, BGK; Critical Reviews- MÇU, HKÜ, SDU, BGK.
Conflict of Interest: The authors declared that there is no conflict of interest..
Financial Disclosure: This work has not received any grants from financial institutions in the public, commercial or not-for-profit sectors.
REFERENCES
- 1.Shaffer D, Pfeffer CR. Practice parameter for the assessment and treatment of children and adolescents with suicidal behavior. J Am Acad Child Adolesc Psychiatry. 2001;40(7):24S–51S. doi: 10.1097/00004583-200107001-00003. [DOI] [PubMed] [Google Scholar]
- 2.Horowitz LM, Bridge JA, Pao M, Boudreaux ED. Screening youth for suicide risk in medical settings:time to ask questions. Am J Prev Med. 2014;47(3):S170–S175. doi: 10.1016/j.amepre.2014.06.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.DeVylder JE, Ryan TC, Cwik M, Wilson ME, Jay S, Nestadt PS, et al. Assessment of selective and universal screening for suicide risk in a pediatric emergency department. JAMA Netw Open. 2019;2(10):1–13. doi: 10.1001/jamanetworkopen.2019.14070. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Ballard ED, Bosk A, Snyder D, Pao M, Bridge JA, Wharff EA, et al. Patients'opinions about suicide screening in a pediatric emergency department. Pediatr Emerg Care. 2012;28(1):34–38. doi: 10.1097/PEC.0b013e31823f2315. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Horowitz LM, Snyder D, Ludi E, Rosenstein DL, Kohn-Godbout J, Lee L, et al. Ask suicide-screening questions to everyone in medical settings:the asQ'em quality improvement project. Psychosomatics. 2013;54(3):239–247. doi: 10.1016/j.psym.2013.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.İncesu E, Beylik U, Küçükkendirci H. Acil servis sağlık hizmetlerinde başvuru tekrarısorunu:Türkiye'de bir devlet hastanesi acil servis araştırması. Akademik BakışUluslararasıHakemli Sosyal Bilimler E-Dergisi. 2016;53:1–13. [Google Scholar]
- 7.Atlı Z, Eskin M, Dereboy Ç. The validity and the reliliability of Suicide Probability Scale (SPS) in clinical sample. J Clin Psy. 2009;12(3):111–124. [Google Scholar]
- 8.Kilincaslan A, Gunes A, Eskin M, Madan A. Linguistic adaptation and psychometric properties of the Columbia-Suicide Severity Rating Scale among a heterogeneous sample of adolescents in Turkey. Int J Psychiatry Med. 2019;54(2):115–132. doi: 10.1177/0091217418791454. [DOI] [PubMed] [Google Scholar]
- 9.Horowitz LM, Bridge JA, Teach SJ, Ballard E, Klima J, Rosenstein DL, et al. Ask Suicide-Screening Questions (ASQ):a brief instrument for the pediatric emergency department. Arch Pediatr Adolesc Med. 2012;166(12):1170–1176. doi: 10.1001/archpediatrics.2012.1276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Horowitz LM, Wharff EA, Mournet AM, Ross AM, McBee-Strayer S, He J-P, et al. Validation and feasibility of the ASQ among pediatric medical and surgical inpatients. Hosp Pediatr. 2020;10(9):750–757. doi: 10.1542/hpeds.2020-0087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Aguinaldo LD, Sullivant S, Lanzillo EC, Ross A, He J-P, Bradley-Ewing A, et al. Validation of the ask suicide-screening questions (ASQ) with youth in outpatient specialty and primary care clinics. Gen Hosp Psychiatry. 2021;68:52–58. doi: 10.1016/j.genhosppsych.2020.11.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Brahmbhatt K, Kurtz BP, Afzal KI, Giles LL, Kowal ED, Johnson KP, et al. Suicide risk screening in pediatric hospitals:clinical pathways to address a global health crisis. Psychosomatics. 2019;60(1):1–9. doi: 10.1016/j.psym.2018.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Thom R, Hogan C, Hazen E. Suicide risk screening in the hospital setting:a review of brief validated tools. Psychosomatics. 2020;61(1):1–7. doi: 10.1016/j.psym.2019.08.009. [DOI] [PubMed] [Google Scholar]
- 14.Roaten K, Horowitz LM, Bridge JA, Goans CR, McKintosh C, Genzel R, et al. Universal pediatric suicide risk screening in a health care system:90,000 patient encounters. J Acad Consult Liaison Psychiatry. 2021;62(4):421–429. doi: 10.1016/j.jaclp.2020.12.002. [DOI] [PubMed] [Google Scholar]
- 15.Akgül A. Tıbbi araştırmalarda istatistiksel analiz teknikleri:SPSS uygulamaları. Ankara: Emek Ofset; 2005. [Google Scholar]
- 16.Cull JG, Gill WS. Suicide probability scale. J Consult Clin Psychol. 1988 [Google Scholar]
- 17.Larzelere RE, Smith GL, Batenhorst LM, Kelly DB. Predictive validity of the Suicide Probability Scale among adolescents in group home treatment. J Am Acad Child Adolesc Psychiatry. 1996;35(2):166–172. doi: 10.1097/00004583-199602000-00009. [DOI] [PubMed] [Google Scholar]
- 18.Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40(11):1337–1345. doi: 10.1097/00004583-200111000-00015. [DOI] [PubMed] [Google Scholar]
- 19.Güvenir T, Özbek A, Baykara B, Arkar H, Şentürk B, İncekaş S. Güçler ve güçlükler anketinin (GGA) Türkçe uyarlamasının psikometrik özellikleri. Turk J Child Adolesc Ment Health. 2008;15:65–74. https://cms.galenos.com.tr/Uploads/Article_30252/cogepderg-15-65.pdf. [Google Scholar]
- 20.Kovacs M. Rating scales to assess depression in school-aged children. Acta Paedopsychiatrica: Int J Child Adolesc Psychiatry; 1981. [PubMed] [Google Scholar]
- 21.Öy B. Çocuklar için depresyon ölçeği:Geçerlik ve güvenirlik çalışması. Türk Psikiyatri Derg. 1991;2(2):132–136. [Google Scholar]
- 22.Huth-Bocks AC, Kerr DC, Ivey AZ, Kramer AC, King CA. Assessment of psychiatrically hospitalized suicidal adolescents:self-report instruments as predictors of suicidal thoughts and behavior. J Am Acad Child Adolesc Psychiatry. 2007;46(3):387–395. doi: 10.1097/chi.0b013e31802b9535. [DOI] [PubMed] [Google Scholar]
- 23.Lanzillo EC, Horowitz LM, Wharff EA, Sheftall AH, Pao M, Bridge JA. The importance of screening preteens for suicide risk in the emergency department. Hosp Pediatr. 2019;9(4):305–307. doi: 10.1542/hpeds.2018-0154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Ballard ED, Cwik M, Van Eck K, Goldstein M, Alfes C, Wilson ME, et al. Identification of at-risk youth by suicide screening in a pediatric emergency department. Prev Sci. 2017;18(2):174–182. doi: 10.1007/s11121-016-0717-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Kim M-J, Paek S-H, Kwon J-H, Park S-H, Chung H-J, Byun Y-H. Changes in Suicide Rate and Characteristics According to Age of Suicide Attempters before and after COVID-19. Children. 2022;9(2):151–159. doi: 10.3390/children9020151. [DOI] [PMC free article] [PubMed] [Google Scholar]