Translation, validity and reliability of decision style scale in forensic psychiatric setting in Indonesia

Abstract

Objective

Clinical reasoning as an essential skill for psychiatrists, especially in forensic psychiatry, relies on their thinking and decision-making skills. However, not all psychiatrists are aware of their decision-making styles. This study examines the validity and reliability of the Indonesian translation of the Decision Style Scale (DSS) instrument among general psychiatrists in the Indonesian forensic psychiatry setting.

Method

This is a cross-sectional study involving 32 general psychiatrists from all nine psychiatric residency training centers in Indonesia. The study was conducted between August 2020 and February 2021. The translation process involved certified independent translators. The validity was tested using Item-Level Content Validity Index (I-CVI), Scale-Level Level Content Validity Index (S-CVI), and item-total correlation. Internal consistency reliability was measured using Cronbach's alpha.

Results

After translation, the instrument was sent back and received feedback from the original authors of DSS. The final version of DSS was valid with an I-CVI score of 0.84–1 and an S-CVI score of 0.99. All but one item had a corrected item-total correlation of more than 0.30. The reliability test of DSS also showed acceptable results with Cronbach's alpha values of 0.43–0.83, and an overall internal consistency score of 0.83 and 0.62 for intuitive and rational scales, respectively.

Conclusion

DSS serves as a valid, reliable, and readily-available tool to measure psychiatrists' decision-making styles in forensic psychiatry settings. Enhancing psychiatrists’ awareness of their decision-making styles may help in mitigating the risk of bias in forensic psychiatry evaluations.

Decision-making; Forensic psychiatry cases; General psychiatrists; Validity; Reliability.

1. Introduction

Clinical reasoning is an essential skill for psychiatrists, especially in forensic psychiatry settings, as they are expected to master the knowledge base and to organize and utilize that knowledge to come to a logical and justifiable conclusion. The dual-process theory posits that, clinical reasoning relies on an individual's thinking and decision-making skills. According to the theory, individuals make decisions in two distinctive ways - fast, implicit, intuitive vs slow, explicit, rational - or Type 1 and Type 2 processing, respectively, and psychiatrists are no exception. Type 1 processing reaches a decision through intuition, while Type 2 processing utilizes higher-order cognitive functions to reach a decision by logically analyzing information. Thus Type 1 processing is deemed prone to bias, neglecting ambiguity, focusing mainly on existing evidence, and ignoring absent evidence [1,2].

In forensic psychiatry, bias in formulating a forensic psychiatry report may lead to harmful short- and long-term consequences for all parties involved. Nevertheless, Type 1 processing cannot be eliminated from the clinical reasoning process, only monitored and managed by Type 2 processing so as to minimize the effect of bias in the final decisions. This would be even more urgent for psychiatrist that tend to rely on Type 1 processing. However, not all psychiatrists are aware of their decision-making styles, leaving them vulnerable to unchecked cognitive bias [1,2].

Objective measurement of decision styles can aid in educating psychiatrists about their decision styles and provide information for faculties to create interventions and further studies aiming to improve the quality of forensic psychiatry assessments. The Decision Style Scale (DSS) is a questionnaire developed by Hamilton et al. consisting of 10-items representing rational and intuitive dimensions to assess how individuals make their decisions [3]. The reliability and dimensionality of the DSS have been evaluated, revealing that the scale has a clear factor structure and high internal consistency. This questionnaire has been used in several studies and has been translated and adapted cross-culturally in other populations with excellent goodness of fit and high reliability [4]. However, this instrument has not yet been studied and utilized in forensic psychiatry and the Indonesian population. This study aims to determine the validity and reliability of the Bahasa Indonesia version of the Decision Style Scale questionnaire, particularly in conducting forensic psychiatric evaluations in the Indonesian psychiatric population.

2. Materials and methods

2.1. Study design

This cross-sectional study is part of more extensive research regarding clinical reasoning skills in forensic psychiatric practice in Indonesia. The study was conducted between August 2020 and February 2021 using online forms distributed through email and phone numbers. Ethical approval was obtained from the Health Research Ethics Committee, Faculty of Medicine, University of Indonesia (KET-571/UN2.F1/ETIK/PPM.00.02/2020).

2.2. Participants

Based on the findings in a systematic review done by Wei et al. regarding the quality of a good psychometric study, a sample size of ≥30 was proposed to examine the reliability and validity of the DSS [5]. Participants were chosen through purposive sampling from a list attained through the Indonesian Psychiatric Association (PDSKJI). Inclusion criteria were actively practicing psychiatrists in PDSKJI that have written at least one forensic psychiatry report in the last three years. Forensic psychiatry consultants, forensic psychiatry fellows, and those with a history of neuropsychiatric disturbances were excluded from the study. Eligible subjects were contacted to participate in the study.

2.3. The Decision Style Scale

The Decision Style Scale (DSS) was developed and validated by Hamilton et al. in 2016 as a self-rated instrument to measure decision styles which reflect the typical manner of individual decision making [3]. It consists of rational and intuitive domains, with five items representing each domain. Respondents rate each item with a 5-point Likert scale ranging from 1 (strongly disagree), 2 (disagree), 3 (neutral, neither agree nor disagree), 4 (agree), and 5 (strongly agree). Numbers in each domain are added, with higher numbers depicting a higher tendency to use the corresponding decision style. The final score was assessed as a numeric outcome with no cutoff score since the authors of DSS viewed the rational and intuitive styles as independent but correlated decision styles, not as opposite ends of the same spectrum. An individual could be strong on one style and weak on the other, strong on both or weak on both. The scale was developed for a general population, and studies regarding its usage on specific populations such as psychiatrists are limited.

2.4. Translation

After obtaining permission from its developers, the DSS was translated from its original English language to Bahasa Indonesia by two independent translators from a licensed translation service. The authors then combined the two translations into the first translated draft. The first translated draft was translated back into English by two other independent translators who had never read the original instrument. The results were then combined into a back-translated draft. Subsequently, the authors sent the first translated draft and its back-translation to the original developers. Further revisions were conducted following feedback. The final translation was used after receiving approval.

2.5. Validity

2.5.1. Content validity

Scale-Level Content Validity Index (S-CVI) were calculated to demonstrate content validity. After giving informed consent, participants were asked to rate each item in the questionnaire on a 4-point Likert scale: 1 (highly irrelevant), 2 (not relevant), 3 (relevant), and 4 (highly relevant). Scores 1 and 2 were then classified as irrelevant (value of 0), while scores 3 and 4 as relevant (value of 1). S-CVI/Ave was calculated to determine each item's mean I-CVI. The measurement of content validity depicts the degree to which an item in an assessment instrument is relevant to a representative for a particular purpose. The higher a content validity test, the more accurate it is in measuring the target construct. I-CVI > 0.79 means that the item is relevant and does not need further revision and S-CVI/Ave ≥0.9 indicates that the items have excellent content validity [6, 7, 8].

2.5.2. Construct validity

Measurement of construct validity was attained by calculating corrected item-total correlation, which represents the correlation between a given item and all other items. Item-total correlation of 0.30 and 0.49 indicates a medium correlation, while higher than 0.50 depicts a strong correlation [9]. Content and construct validity were analysed using IBM SPSS Statistics 25.

2.5.3. Reliability

The instrument's reliability was established using Cronbach's alpha measurement to demonstrate internal consistency. An item is considered reliable with Cronbach's alpha score greater than 0.6, acceptable between 0.6 to 0.8, with a corrected item-total correlation greater than 0.3 [9, 10]. Data analysis was also done in IBM SPSS Statistics 25.

3. Results

3.1. Descriptive findings

Thirty-two psychiatrists from all nine residency training centers in Indonesia agreed to participate in the study and signed an online informed consent. Members of this study consisted of 21.9% males and 78.1% females (Table 1). The majority of participants have had a history of acquiring a forensic psychiatry module during residency and a form of forensic psychiatry training.

Table 1.

Demographic information.

Characteristic	Mean ±Standard Deviation	Frequency (n = 32)	Percentage
Age	46.34 ± 9.9
Sex  Female  Male		23 9	71.9 28.1
Psychiatry Residency Training Center Background (Province)  DKI Jakarta  D.I. Yogyakarta  Central Java  West Java  Central Java  East Java  Bali  North Sumatera  South Sulawesi		7 6 5 5 3 3 1 1 1	21.9 18.8 15.6 15.6 9.4 9.4 3.1 3.1 3.1
History of acquiring forensic psychiatry module during residency  Yes  No		26 6	81.3 18.7
History of forensic psychiatry training  Yes  No		27 5	84.4 15.6
History of negative experience in conducting forensic psychiatric evaluations  Yes  No		13 19	40.6 59.4

3.2. Translation

After receiving permission and agreeing upon the translated Bahasa Indonesian version of DSS, the instrument was modified based on the feedback from DSS instrument's original creators, as seen on Table 2.

Table 2.

Feedback from the original creator of DSS and final revision of Bahasa Indonesia DSS.

Item	Original Version	Translated Version	Backward Translated Version	Feedback from Author	Final Revision
Rational Questions
2	I thoroughly evaluate decision alternatives before making a final choice	Saya mengevaluasi semua alternatif keputusan secara menyeluruh sebelum pada akhirnya mengambil keputusan akhir	I evaluate all decision alternatives thoroughly before I finally make a final decision	“I am concerned with the addition of the word “finally” in items 2 and 5 of the rational scale. It places a heavier emphasis on the temporal aspect of decision making than there should be.”	Saya mengevaluasi semua alternatif keputusan secara menyeluruh sebelum mengambil keputusan akhir
5	I weigh a number of different factors when making decisions	Saya menimbang sejumlah faktor yang berbeda sebelum akhirnya mengambil keputusan	I weigh many different factors before I finally make a decision.	“I am concerned with the addition of the word “finally” in items 2 and 5 of the rational scale. It places a heavier emphasis on the temporal aspect of decision making than there should be.”	Saya menimbang sejumlah faktor yang berbeda sebelum mengambil keputusan.
Intuitive Questions
1	When making decisions, I rely mainly on my gut feelings	Pada saat mengambil keputusan, saya terutama mengandalkan intuisi saya	When I make a decision, I mainly rely on my intuition	“In the intuitive items, for item 1, intuition should really be captured by a word representing feelings. My co-authors and I have discussed the intuition scale being tied into how someone feels about the choice not so much thinks.”	Pada saat mengambil keputusan, saya terutama mengandalkan insting/perasaan saya

3.3. Validity of decision style scale

3.3.1. Content validity

The internal validity test for the rational scale in DSS showed I-CVI in the range of 0.94–1.0, with the S-CVI/Ave of 0.99. The result indicates that the participants deemed 99% of the items in the rational scale to be relevant and clear. On the other hand, I-CVI for the intuitive scale of DSS ranged from 0.84 to 1. S-CVI/Ave of the intuitive scale was 0.93, meaning 93% of the items were unambiguous, clear, and relevant to the study participants. Results for the content validity of DSS are presented in Table 3. Each item was shown to have good content validation.

Table 3.

Validity test of Bahasa Indonesian version of Decision Style Scale.

Rational Scale			Intuitive Scale
Item	Relevancy	I-CVI	Item	Relevancy	I-CVI
1	32	1	6	32	1
2	32	1	7	31	0.97
3	30	0.94	8	30	0.94
4	32	1	9	29	0.91
5	32	1	10	27	0.84
S-CVI/Ave = mean I-CVI = 0.99			S-CVI/Ave = mean I-CVI = 0.93

Relevancy, number of experts who judged the item to be relevant according to the measurement objectives.

I-CVI, item-level content validity index.

S-CVI, scale-level content validity index.

3.3.2. Construct validity

Corrected item-total correlation depicts the correlation between a given item and the sum score of other items. A score above 0.5 means that a strong, positive correlation is found between the scores, while a score between 0,3-0,5 was considered acceptable. In this study (Table 4), seven out of ten items had a corrected item-total correlation of more than 0.5, while three items (Item 1, 3, and 5) in the rational scale of DSS had a corrected item-total correlation of 0.44, 0.11, and 0.42, respectively. The i tem with a 0.11 item-total correlation score was rephrased for clarity. Both authors and the expert panel decided to retain the item since the item was considered an essential aspect in the forensic psychiatry setting.

Table 4.

Internal consistency coefficient (α) and corrected item-total correlation for Bahasa Indonesia version of DSS.

Scale	Cronbach's alpha (α)	Corrected item- total correlation
Rational Scale
Total	0.62
Item 1	0.56	0.44
Item 2	0.43	0.66
Item 3	0.83	0.11
Item 4	0.47	0.67
Item 5	0.55	0.42
Intuitive Scale
Total	0.83
Item 1	0.81	0.58
Item 2	0.80	0.63
Item 3	0.78	0.69
Item 4	0.78	0.69
Item 5	0.81	0.58

3.4. Reliability of decision style scale

Cronbach's alpha (α) is a measurement that calculates the internal consistency of an assessment instrument. The value of Cronbach's alpha between 0.6 to 0.8 is deemed acceptable [8]. In this study, the Cronbach's alpha of the intuitive scale all scored above 0.6 (0.78–0.81) with an overall internal consistency of 0.83. Meanwhile, the rational scale had an α ranging from 0.43 to 0.83, with an overall internal consistency of 0.62. Both scales had acceptable overall internal consistency.

4. Discussion

This study intends to evaluate the validity and reliability of the Decision Style Scale for use by psychiatrists in Indonesia, particularly in conducting forensic psychiatry evaluations. Decision styles are independent of cognitive abilities; it is influenced by personal characteristics of the decision-maker such as upbringing, personality, values, motivations, tolerance to risk and uncertainty, and influence from environmental and situational factors [2, 3]. Previous studies have shown that decision styles can predict self-ratings of decision quality better than Big Five personality traits [11]. Although an individual may use multiple decision styles, each person has a dominant style.

Numerous typologies of decision styles have been proposed, and the instruments to measure it have been developed. However, these studies’ instruments lack construct validity evidence and strong psychometric measurements. Additionally, few are readily accessible, widely generalizable, and easy to use [3].

The Decision Style Scale is a valid and reliable tool that measures two distinct decision styles. It was developed to fill the necessity of systematic and theory-driven measurement of decision styles and encourage research domains where decision styles are essential. It can also be used in training programs to elucidate personal decision styles and provide feedback for improvement based on the suitable style in a given setting [3]. The instrument has been developed with undergraduate students as the sample population and has been further used in numerous studies. Among others, it has been used in the context of criminal choice and decision making in tort cases [12, 13]. And so, we believe that this tool has the potential to be used in the Indonesian forensic psychiatry setting, where decision styles and cognitive processing are vital in producing quality assessments, reports, and testimony.

Two typologies measured in DSS are rational and intuitive mode. Rational mode is described as “intentional, analytic, relatively slow, rule-governed, and logically defensible”. On the opposite, the intuitive mode is defined as “automatic, preconscious, relatively fast, affect-laden, heuristic, and experience-based” [3]. It refers to the dual-process framework of type 1 and type 2 processing, which has also been linked as a clinical reasoning process applicable to conducting forensic psychiatry evaluations, with type 1 processes characterized as autonomically activated in concordance to relevant stimuli (intuitive), and type 2 as engaging higher-order cognitive functions, and it uses a slower but more meticulous reasoning (rational) [2, 14].

In conducting forensic psychiatry assessments, the combination of both Type 1 and Type 2 processes is thought to be superior compared to either strategy alone [15]. However, type 2 is especially important as it mitigates bias by combining numerous data and logical pathways to arrive at an assessment. It plays as a gatekeeper for hypotheses generated by type 1 processing [2]. Measuring decision style scales in forensic psychiatry settings may provide insight into current decision style practices in Indonesian psychiatrists and also assist in determining appropriate ways to educate for better practice. Thus, having a valid and reliable questionnaire to determine decision styles is necessary.

This study indicates that the instrument had excellent content validity, with I-CVI of 0,84-1 and S-CVI of 0.99 and 0.93 on the rational and intuitive scales. Additionally, construct validity as measured with corrected item-total correlation revealed seven out of ten items having strong correlation, two items on the rational scale having medium correlation, and one item from the rational scale with <0.30 corrected item-total correlation (Item 3). This item was not eliminated as it was deemed an essential aspect of rational decision making. The instrument's reliability measured with Cronbach's alpha was acceptable, with an overall internal consistency of 0.83 on the intuitive scale and 0.62 on the rational scale. Each item in the intuitive scale was >0.6, while several items in the rational scale were less than 0.6. This promising pilot study can benefit from a larger-scale study, as a more accurate estimate of the population Cronbach's alpha can be obtained from larger samples [16].

4.1. Future implication

Studies regarding decision styles in forensic psychiatrists are still lacking. The translated and validated DSS instrument can be used to evaluate decision styles in psychiatrists, particularly in conducting forensic psychiatry assessments. Information regarding decision styles in psychiatrists can then be used to support correlations with the quality of forensic psychiatry reports and assessments. Psychiatrists and educators may benefit from this insight into their decision-making style, and insight into creating targeted interventions to increase the quality of forensic psychiatry assessments. Currently, this study facilitates the self-measurement of decision-making styles as one of the internal factors of psycho-medico-legal analysis skills that will be assessed in a more extensive ongoing study: “Development of Forensic Psychiatry Assessment Instrument and Module: Focusing on Psycho-medico-legal Analysis Thinking Skills among Indonesian Psychiatrists”. Although this study is focused on the use of DSS among psychiatrists in Indonesia, this study can be used as a stepping stone for the implication of this instrument in forensic psychiatric settings worldwide.

4.2. Study limitations

The limitations of this study are the limited number of participants and the recruitment method. Samples were chosen by purposive sampling on the national psychiatric database, with unequal distribution of geographic region and residency training background of the participants. Further studies may benefit from higher sample sizes with stratified random sampling from each region and residency training background. Additionally, repeated measurements can be done to assess the test-retest reliability of the DSS, which has not been done in this study.

5. Conclusion

Decision styles play an integral part in the clinical reasoning applied in forensic psychiatry assessments. This study is the first to translate and measure the validity and reliability of the Decision Style Scales in Bahasa Indonesia, revealing a valid and reliable instrument. This instrument can be further used to study decision styles in psychiatrists and how it affects forensic psychiatry assessments.

Declarations

Author contribution statement

Natalia Widiasih Raharjanti; Monika Kristi Levania: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Tjhin Wiguna; Agus Purwadianto; Diantha Soemantri; Wresti Indriatmi; Elizabeth Kristi Poerwandari; Marlina S.Mahajudin: Conceived and designed the experiments; Wrote the paper.

Nadia Rahmadiani Nugrahadi; Aisha Emilirosy Roekman: Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Olivia Jeany Darmawan Adji Saroso: Performed the experiments; Wrote the paper.

Adhitya Sigit Ramadianto: Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

Mrs. Natalia Widiasih Raharjanti was supported by PUTI Grant [NKB- 570/UN2.RST/HKP.05.00/2020].

Data availability statement

Data included in article/supp. material/referenced in article.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.