Psychometric accuracy of the Dutch Child and Adolescent Trauma Screener

ABSTRACT

Objective: The aim of this study is to investigate the psychometrics of the Dutch version of the Child and Adolescent Trauma Screener (CATS-2). By this, an international recognized instrument to screen symptoms of post-traumatic stress (PTSS) in children and adolescents according to the Diagnostic and Statistical Manual for Mental Disorders, 5th edition (DSM-5) becomes available for Dutch youth.

Procedure and Method: Based on the validated CATS-2 we established the Dutch version, named the KJTS. A total of 587 children and adolescent, age 7–21, and 658 caregivers referred to mental health care services in Amsterdam was included in the study to examine psychometric properties. The construct was tested by confirmatory factor analysis (CFA). Furthermore reliability, convergent-divergent patterns and diagnostic test accuracy were examined.

Results: The underlying DSM-5 factor structure with four symptom clusters (re-experiencing, avoidance, negative alterations in mood and cognitions, hyperarousal) was supported by CFA showing a good fit for the selfreport (CFI = .95, TLI = .94), and an acceptable fit for the caregiver report (CFI = .90, TLI = .89). The KJTS showed excellent reliability (alpha = .92) on both selfreport and caregiver report. The convergent-discriminant validity pattern showed medium to strong correlations with measures of internalization problems, such as anxiety and affective problems (r = .44–.72) and low to medium correlations with externalizing symptoms (r = .21–.36). The ROC-curve analysis has proven a good accuracy (AUC = .81; n = 106).

Discussion and conclusion: This study demonstrates the psychometric accuracy of the KJTS in a Dutch clinical population. The KJTS reflects adequately the dimensionality of PTSD as described in the DSM-5, with a good fit for selfreports, an acceptable fit for caregiver reports, excellent reliability and sufficient validity. Limitations are described. The outcomes support the use of the KJTS in research and clinical practice for screening and monitoring of PTSS.

HIGHLIGHTS

• The Dutch version of the Child and Adolescent Trauma Screener (KJTS) is a questionnaire for the assessment of the risk on PTSD in children and adolescents. It assists the mental health care professional to estimate the prevalence and severity of potentially traumatic events and posttraumatic stress symptoms.

• The KJTS consists of a selfreport (7+ years) and a caregiver report, both questionnaires show good psychometric accuracy, in line with international findings.

• The KJTS is a freely accessible, easy to use and license-free instrument for mental health professionals.

1. Introduction

Posttraumatic stress symptoms (PTSS) can occur after experiencing one or multiple traumatic events. Prevalence rates for traumatic events before the age of 18 years range from 30% up to 70%, depending on the type of adverse event recorded and the income level of a country (Briggs-Gowan et al., 2010; Copeland et al., 2007; Costello et al., 2005). These children and adolescents are potentially at risk of developing Post Traumatic Stress Disorder (PTSD), with an average prevalence rate of 16% after undergoing an traumatic event (Alisic et al., 2014). Additionally, going through multiple traumatic events augments the risk of developing PTSS and lifetime PTSD in adulthood (da Silva et al., 2024). Alongside the impact of PTSS, these children and adolescents are vulnerable to a range of other severe mental health disorders such as anxiety disorders, depression, substance abuse and behavioural problems (Bastien et al., 2020; Jonkman et al., 2013; Smith et al., 2019; Teicher & Samson, 2013). Not merely the child and their surroundings are affected by PTSS, the problems also extend into society. Studies show that PTSD induces a significant economic burden, estimated that societal costs are up to three times higher for individuals with PTSD than for non-exposed individuals (Bothe et al., 2020; Davis et al., 2022). Thus, children and adolescents with PTSD are at increased risk of negative outcomes, including financial challenges, academic difficulties, social isolation, and impaired relationships and engagement in age-appropriate activities (Cohen et al., 2002). Consequently for both the individual child and society it is vital to effectively assess and treat PTSD.

In recent decades, numerous effective interventions targeting PTSD have evolved (Bastien et al., 2020). To begin these interventions as early as possible, adequate screening for PTSD in children and adolescents is essential for identification, intervention, and support, aiming to improve their immediate and long-term mental health outcomes. Identifying PTSD and offering appropriate treatment plays a crucial role in breaking the cycle of intergenerational transmission of trauma and fostering resilience in children who have experienced traumatic events (Perrin et al., 2005).

1.1. Classification of PTSD

In order to recognize PTSD, it is essential to have a framework mapping the symptoms. The need for a classification of mental disorders led to the creation of the Diagnostic and Statistical Manual of Mental Disorders (DSM) in the early 1950s. In the late twentieth century, PTSD was added as a disorder to the third edition of the DSM (American Psychiatric Association, 1980, DSM-III), based on PTSD as seen in adult patients. In this third edition, and later in the fourth edition of the DSM, PTSD was grouped as a fear-based anxiety disorder. In 2013 the latest revision of the DSM (DSM-5) made notable changes with both important conceptual and clinical implications, and PTSD is currently grouped under the Trauma- and Stressor-Related Disorders. The diagnostic criteria of PTSD have been modified, based on the emerging research evidence and knowledge about PTSD in children and adolescents. In the current classification, the onset of PTSD must be preceded by exposure to a traumatic or adverse event. Symptoms following these events are defined in four clusters, with 20 symptoms, in youth from age 6 till adulthood, and 16 symptoms for preschool children (American Psychiatric Association, 2013). Changes between DSM-IV and DSM-5 impact the applicability of existing assessment tools, which are based on the fourth edition, such as the CRIES-13 questionnaire for children (Verlinden et al., 2014). Three years after the DSM-5 release, a validated and suitable trauma screening instrument was created for adult mental healthcare (Bovin et al., 2016). In youth, however, a validated and appropriate assessment tool was not yet available.

1.2. Screening PTSD in youth

To address this need, in 2017 a DSM-5 PTSD screening questionnaire for children and adolescents was created in different languages and validated in Germany, the UK and Norway: The Child and Adolescent Trauma Screener (CATS) (Sachser et al., 2017). The CATS is a short questionnaire to screen for exposure to adverse and potentially traumatic events and PTSS. A selfreport and a caregiver report are available for children and adolescents ages 7–18 years, and a caregiver report for children ages 3–6 years. Items are directly based on the criteria of PTSD according to the DSM-5: criteria A (exposure to a traumatic event), and the four clusters of symptoms: criteria B (intrusions), C (avoidance), D (negative alterations in cognitions and mood) and E (hyperarousal). The initial validation study of the CATS was conducted using samples from the United States, Norway and Germany (Dowdy-Hazlett et al., 2021; Sachser et al., 2017). In this validation process, adaptations were made, and in 2019 the CATS-2 was established and subsequently validated in 2022 (Sachser et al., 2022). Improvements in the CATS-2 focused on language changes, such as separating long questions in multiple shorter questions, and changes in the list of PTEs. In the Netherlands, we translated the CATS-2 in Dutch: the Kind en Jeugd Trauma Screener (KJTS). The KJTS allows us to identify symptoms, explore prevalence rates of PTSS and monitor symptoms over time or throughout treatment. Such an assessment can be used in all fields of child and adolescent mental healthcare to promptly asses the risk of PTSD and estimate the prevalence and severity of traumatic events and symptoms. It helps mental health professionals to systematically ask about, and identify trauma-related problems underlying common behaviours such as concentration problems, sleeping problems, aggressive behaviour or negative thoughts.

It is crucial to establish the psychometric accuracy of an assessment, to ensure a valid questionnaire. Thereupon, this questionnaire can be used as a suitable measure in clinical practice and in further clinical research (Dowdy-Hazlett et al., 2021). Children’s perspective and caregivers’ perspectives are included, providing valuable insights into the (dis)agreements between the child and caregiver. Both reports should be included in a trauma screening procedures in mental health care to create a complete picture of the PTE’s and PTSS of the child (Skar et al., 2021). The aim of the present study is to assess the psychometric accuracy of the KJTS selfreport and KJTS caregiver report for children and adolescents age 7 years and above, in order to establish a standardized selfreport and proxy assessment. This psychometric accuracy will be assessed using confirmatory factor analysis (CFA) to investigate whether the four clusters (B, C, D and E) yield a good fit for the four-factor structure of the DSM-5. Furthermore we will examine reliability, internal consistency and the convergent-discriminant validity pattern, and diagnostic test accuracy.

We expect to find a good fit for the four clusters of the KJTS, since it is based on the classification system (DSM-5; American Psychiatric Association, 2013), and other previous studies confirmed this four-factor structure as an adequate fit for the underlying DSM-5 structure (Dowdy-Hazlett et al., 2021; Hafstad et al., 2014; Sachser et al., 2017). Furthermore we expect an accurate reliability based on the α ranging between .88 and .94 for the CATS-2 in neighbouring countries (Nilsson et al., 2021; Sachser et al., 2022). Also, we expect in our divergent-convergent pattern, a strong positive correlation with internalizing problems, such as anxiety and depressive symptoms, and a lower positive to no significant correlation with externalizing behaviour. The correlation between PTSD symptoms and internalizing problems is complex and varies among individuals, but it is common for individuals with PTSD symptoms to experience internalizing symptoms, such as anxiety, sleep disturbances, negative feelings and changes in mood (Campbell et al., 2007; Lecrubier, 2004). We expect a smaller correlation with externalizing symptoms. While there is overlap in symptoms, such as risk seeking, concentration problems, and irritability, externalizing problems and PTSD represent distinct clinical entities with different diagnostic criteria. This aligns with previous studies of the CATS, which showed medium to strong correlations with measures of depression (r = .62–.82) and anxiety (r = .40–.77) and low to medium correlations with externalizing symptoms (r = –.15–.43) (Nilsson et al., 2021; Sachser et al., 2017; Sachser et al., 2022). Finally, we expect good diagnostic test accuracy (Sachser et al., 2022).

2. Methods

2.1. Establishing the KJTS

The KJTS is established through a back-to-back translation process. Firstly, the German version was translated to Dutch by a Dutch/German bilingual professional. Subsequently, this Dutch version was evaluated by five independent clinicians and researchers. These clinicians read through both the events and the symptoms list. Minor language and lay out adaptions were done, such as adding an option ‘cannot say’ to the list of PTE’s. This option has been added for participants whom are not able (yet) to speak about what happened to them, but do want to state a potentially traumatic event has happened to them. Thirdly the evaluated Dutch version was translated back to German by an independent certified German/Dutch translator. In the next step, we presented this German version to the original research group who developed the CATS-2 (Sachser et al., 2022). Minimal language changes were made, such as changing the word nightmares to bad dreams in the Dutch language. Successively, the face validity was tested through the Thinking Out Loud Method (Boren & Ramey, 2000). This method is used to estimate if the questions are clear and the participant understands the question in a correct way. Five children aged 9 to 20 years, and three caregivers were asked to fill out the form while speaking out loud their thoughts with the researcher present. Exclusively motivation to voice their thoughts was given, without additional information or questions. No extra adaptations were needed. Consequently, the KJTS was implemented in care as usual in the department of trauma attachment and family, child- and adolescent psychiatry clinic in Amsterdam The Netherlands. Due to positive response in this department, the KJTS was included in the online questionnaire package of the standardized intake that is send out to every referred child and their caregiver (September 2021). The decision to include the KJTS in the standardized intake, was supported by the fact that the questionnaire is well established in Germany, Norway and the U.K. and is implemented in some of the usual care, due to good psychometric properties. An illustrated overview of the implementation process of the KJTS is displayed in Figure 1. The third step is what we aim for in current study.

Figure 1.

An overview of implementation steps towards a validated KJTS.

2.2. Procedure

From September 2021 till September 2023 data was collected at Levvel, Child and Adolescent Mental healthcare in Amsterdam, the Netherlands. The KJTS was included as standard care in the Routine Outcome Monitoring (ROM), and was sent out in the standard set of questionnaires for parents and children during their intake for both outpatient and inpatient clinics. This standard set includes the Child Behaviour Checklist (CBCL) for the caregivers and Youth Selfreport (YSR) for the children. This study has been approved by the ethical review board of Amsterdam UMC in November 2019. Since it was performed as a retrospective file study, no informed consent was needed for anonymous use of data. As part of the standard care, the Clinician Administrated PTSD Scale – Children and Adolescents (CAPS-CA) was assessed by trained clinicians, in case there was an indication of risk on PTSD. CAPS-CA data of 106 children and adolescents was available for the sensitivity/specificity analyses in the current study.

2.2.1. Participants

Children, adolescents (age 7–21 years) and their caregiver(s) in the current sample are referred to child-and adolescent psychiatry for a broad array of mental health problems. The sample consists therefore not only off participants with PTSD, but among other things anxiety- and depressive symptoms, child–parent relation problems and behaviour problems. All referred children and their caregiver(s) received the KJTS in the intake routine. Whenever a participant did not confirm any potentially traumatic event, the questionnaire was not continued. All data consists therefore of participants with at least one PTE. An overview of participant characteristics is presented in Table 1.

Table 1.

Participant characteristics.

	KJTS 7+ Selfreport	KJTS 7+ Caregiver report
N	587	658
Mean age of the child (SE)	14.71 (3.23)	12.81 (3.06)
Gender of the child %
Male	29.1	41.6
Female	70.9	58.4
KJTS total score (SE)	27.61 (12.52)	19.48 (11.38)
Traumatic events %
Single	17.2	33.6
Multiple (2–4)	25.7	39.8
Multiple (4+)	57.1	26.6
Relation to the child (N)
Mother	-	474
Father	-	89
Foster mother	-	60
Foster father	-	10
Other	-	25

In Levvel, a Child and Adolescents Mental healthcare service in Amsterdam, children and adolescents until the age of 21 are seen. For this study, all referred children and adolescents of age 7 and above were eligible. The KJTS is a DSM-5-based questionnaire, and since there is no difference in the DSM-5 criteria for PTSD in children 7–18 and 18–21, the same questionnaire is applicable for the whole age range. Therefore we used the complete range of age of children and adolescents who are in treatment at Levvel.

2.2.2. Measures

2.2.2.1. Child and Adolescent Trauma Screener (KJTS)

The KJTS is a short, freely accessible questionnaire to screen for exposure to 20 potentially traumatic events (PTEs) and the 20 PTSD symptoms included in DSM-5. The items directly map onto the clusters B (intrusions), C (avoidance), D (negative alterations in cognitions and mood) and E (hyperarousal). An overview of the KJTS PTEs in Table 2, and the symptom item wordings, means, standard deviations and corrected total item-scale correlations from the selfreport and caregiver report are listed in, Tables 3 and 4. If at least one PTE is evident, posttraumatic stress symptoms (PTSS) are measured by 20 items rated on a scale with the following anchors: 0 = ‘Never’, 1 = ‘Sometimes’, 2 = ‘Often’ and 3 = ‘Almost always’. Psychosocial functioning is assessed using five yes/no items that ask whether the previously rated posttraumatic stress symptoms interfere with five key areas of functioning (getting along with others, school/work, hobbies, family relationships, and general happiness). The list of PTE’s of the KJTS diverts slightly from the PTE checklist of the CATS-2, due to a Dutch PTE checklist that was already implemented in the Dutch mental healthcare system. The following DSM-5-based symptom checklist is fully corresponding with the original CATS-2 version. Implications of the diverted PTE list are elaborated in the discussion.

Table 2.

List of PTEs and their frequencies.

		Selfreport (N = 587)	Caregiver report (N = 658)
Item		N	N
1	Serious accident or injury like a car/bike crash, dog bite, or sports injury	92	42
2	Fire or explosion	40	17
3	Gun shooting	23	1
4	Being around war or lived in an unsafe neighbourhood	35	9
5	Threatened, hit or hurt badly	279	172
6	Being shouted at by a family member	314	259
7	Seeing someone in my family threatened, hit or hurt badly, or family members hurting each other or breaking things	208	140
8	Out of house placement	128	93
9	Have been through unwanted (sexual) touches, or abuse	254	103
10	Someone made me touch this person when I didn’t	101	24
11	Someone threatened to touch me, but it didn’t happen in the end	103	22
12	As a parent I heard that my child has been through unwanted (sexual) touch, but denies this	–	11
13	Did not have enough water or clothing, or no home	64	37
14	Have been left home alone for a longer period of time without a caregiver	69	36
15	Life threatening disease or scary medical procedure	53	39
16	Someone close to the child dying suddenly or violently	257	216
17	Been bullied	309	259
18	Felt racism	67	24
19	Other	162	228
20	Cannot say	41	29

Table 3.

Item wordings, means, standard deviations and corrected total item-scale correlations of the KJTS (Selfreport) N = 587.

Item		M	SD	ISC	Criterion
1	Upsetting thoughts or pictures about what happened that pop into my head	1.40	.92	.671	B
2	Bad dreams reminding me of what happened.	.81	.95	.565	B
3	Pictures in my head of what happened. Feels like it is happening right now.	1.02	.99	.641	B
4	Feeling very upset when I am reminded of what happened.	1.33	1.10	.642	B
5	Strong feelings in my body when I am reminded of what happened (sweating, heart beating fast, upset stomach).	1.20	1.10	.653	B
6	Trying not to think about what happened. Or to not have feelings about it.	1.73	1.12	.518	C
7	Staying away from anything that reminds me of what happened (people, places, things)	1.46	1.08	.499	C
8	Not being able to remember part of what happened.	1.17	1.07	.260	D
9	Having negative thoughts, such as:				D
	a. I won’t have a good life.	1.07	1.09	.625	D
	b. I can’t trust other people.	1.45	1.18	.664	D
	c. The world is unsafe.	1.20	1.09	.553	D
	d. I am not good enough.	1.38	1.18	.597	D
10	Blaming for the event				D
	a. Blaming myself for what happened.	1.22	1.08	.579	D
	b. Blaming someone else for what happened although it wasn’t their fault.	.39	.71	.172	D
11	Upsetting feelings (afraid, angry, guilty, ashamed) a lot of the time.	1.72	1.01	.707	D
12	Not wanting to do things I used to do.	1.24	1.05	.596	D
13	Not feeling close to people.	1.07	1.02	.607	D
14	Not being able to have happy feelings.	1.07	1.00	.656	D
15	Managing strong feelings				E
	a. It is very hard to calm down when I am upset.	1.50	1.10	.393	E
	b. Feeling mad. Having fits of anger and taking it out on others.	1.33	1.06	.412	E
16	Doing unsafe things.	.73	.90	.467	E
17	Being overly careful (checking to see who is around me).	1.76	1.08	.452	E
18	Being jumpy.	1.34	1.09	.488	E
19	Problems paying attention.	1.83	1.05	.458	E
20	Trouble falling or staying asleep.	1.70	1.09	.454	E

Table 4.

Item wordings, means, standard deviations and corrected total item-scale correlations (ISC) of the KJTS (caregiver report) N = 658.

Item		M	SD	ISC	Criterion
1	Upsetting thoughts or memories about what happened pop into the child’s head. Or the child re-enacting what happened in play	.96	.90	.646	B
2	Bad dreams related to what happened.	.52	.73	.522	B
3	Acting, playing, or feeling as if what happened is happening right now.	.30	.63	.451	B
4	Feeling very upset when reminded of what happened.	.91	.95	.630	B
5	Strong physical reactions when reminded of what happened (sweating, heart beating fast, upset stomach).	.65	.90	.616	B
6	Trying not to think about what happened. Or to not have feelings about it.	1.09	1.07	.541	C
7	Avoiding anything that is a reminder of what happened (people, places, things, situations, talks)	.96	1.07	.640	C
8	Not being able to remember an important part of what happened.	.55	.85	.338	D
9	Having negative thoughts, such as:				D
	a. I won’t have a good life.	.95	1.03	.632	D
	b. I can’t trust other people.	.99	1.00	.661	D
	c. The world is unsafe.	.91	.96	.631	D
	d. I am not good enough.	1.47	1.08	.618	D
10	Blame for the event				D
	a. Blaming self for what happened.	.95	1.03	.495	D
	b. Blaming others for what happened although it wasn’t their fault.	.99	1.00	.332	D
11	Upsetting feelings (afraid, angry, guilty, ashamed) a lot of the time.	.91	.96	.709	D
12	Not wanting to do things he/she used to do. Losing interest in activities he/she used to enjoy	1.47	1.08	.578	D
13	Not feeling close to people.	.95	1.03	.604	D
14	Showing or having less happy feelings	.99	1.00	.540	D
15	Managing strong feelings				E
	a. Having a hard time calming down when upset.	1.29	1.02	.453	E
	b. Being irritable. Or having angry outbursts and taking it out on others.	1.17	1.02	.434	E
16	Risky behaviour or behaviour that could be harmful. Doing unsafe things.	.61	.83	.351	E
17	Being overly alert or on guard.	1.24	1.01	.578	E
18	Being jumpy or easily startled.	.88	.98	.544	E
19	Problems with concentration.	1.69	1.02	.478	E
20	Trouble falling or staying asleep.	1.36	1.12	.507	E

2.2.2.2. Child Behaviour Checklist (CBCL) 6–18 years

The Child Behaviour Checklist is a caregiver report instrument to asses emotional, behavioural, and social problems (Achenbach, 2001). The CBCL consists of 120 items that can be rated on a 0–2 rating scale 0 = not true; 1 = somewhat or sometimes true; 2 = very or often true. CBCL gives a total score on emotional and behaviour problems which can be divided in internalization problems (depressive, anxiety and somatic problems) and externalization problems (aggressive and rule breaking behaviour). Achenbach and Rescorla constructed a scoring system based on five DSM-orientated scales: affective problems, anxiety problems, attention-deficit/hyperactivity (ADHD) problems, oppositional defiant problems (ODD), and conduct problems (CD). Many studies showed good psychometric proportions of the CBCL (Warnick et al., 2008), including the Dutch version (Verhulst et al., 1990)

2.2.2.3. Youth Selfreport (YSR) 11–18 years

The Youth Selfreport (YSR) for ages 11–18 is the selfreport equivalent of the CBCL. The YSR consists of 132 items and is rated on a 0–2 rating scale 0 = not true; 1 = somewhat or sometimes true; 2 = very or often true. Items are also divided into internalization problems (depressive, anxiety and somatic problems) and externalization problems (aggressive and rule breaking behaviour) and the DSM scales affective problems, anxiety problems, attention-deficit/hyperactivity (ADHD) problems, oppositional defiant problems (ODD), and conduct problems (CD) (Achenbach, 2001). The YSR shows sufficient psychometric properties in the Dutch sample (Frank Cornelis Verhulst et al., 1997).

2.2.2.4. The Clinician-Administered PTSD Scale for DSM-5 – Child and Adolescent Version (CAPS-CA-5)

The CAPS-CA-5 is a semi-structured 20-item clinician-administered interview to assess PTSD based on DSM-5 criteria for children and adolescents of age 7 years and older (Pynoos et al., 2015). The assessor combines verbal answers, non-verbal observations and other known information into a frequency and intensity of PTSD symptoms, which results in a severity rating or every DSM-5 symptom (0 = ‘absent’, 1 = ‘mild, 2 = ‘moderate’, 3 = ‘severe’, 4 = ‘extreme’). The scoring follows the DSM-5 criteria for PTSD with a symptom rating of ≥ 2 being indicative of the presence of a symptom. The internal consistency was excellent for the 20-item DSM-5 scale (range 0–80) with α = .90. No Dutch psychometric properties are available at the time of this study.

2.2.3. Data analysis

Participants with complete values for the KJTS were included in the analysis. As a first step, a second-order Confirmative Factor Analysis (CFA) was applied to the KJTS selfreport and caregiver report data, to investigate whether the predetermined four-factor structure of PTSD according to the DSM-5 yielded a good fit. A second-order CFA was done, as the theoretical construct PTSD is a hierarchical model, with the overall PTSD construct as the higher order level and the four sub scales as lower order level. The evaluation of the model fit followed the guidelines of a well or acceptable fitting model proposed by Hu and Bentler (1999) using Yuan-Bentler χ², with comparative fit index (CFI) and Tucker-Lewis index (TLI) values as close as possible to 1, with exceeding values of .90 for an acceptable fit and a preferred value of > .95 for a good fit (Hooper, 2008). Furthermore a standardized root mean square residual (SRMR) and root mean squared error of approximation (RMSEA) values equal to or smaller than .06 is necessary for a good fit, with 0.07 the limit for an acceptable value (Hooper, 2008). Factor loadings will be displayed, indicating if the item is meaningfully related to the hypothesized construct. The closer the loading is to −1 or 1, the better the relation is. However, clear recommendations for cutoff values differ throughout literature, from >0.2 till >0.5 for an acceptable interpretable factor loading (Brown, 2015; Ondé & Alvarado, 2020). Secondly, internal consistency (Cronbach’s alpha) of the self-report and caregiver report was assessed. Thereafter, the intraclass correlation coefficient (ICC) (two-way mixed model with measures of absolute agreement) was calculated to determine agreement between self-report and caregiver report. Interpretations of the ICCs followed the criteria: < .50 poor; .50–.75 moderate; .75–.90 good and > .90 excellent, respectively (Koo & Li, 2016). Child and caregiver agreement tend to be better in older children (Skar et al., 2021). To test this hypothesis in our sample we divided the group in three age categories. The first group is primary school children in age 7–11 years, the second group is the first half of high school (12–15 years) and lastly a third group of adolescents of 16 and older. Furthermore, the convergent and divergent validity has been evaluated with bivariate correlations between internalizing problems and externalizing problems. Lastly sensitivity analyses were done. The diagnostic accuracy was explored using receiver operating characteristics (ROC) analyses in the self-report questionnaires. The CAPS-CA-5 was used as the gold standard for PTSD diagnosis. We followed the cut off scores from the Norwegian, U.K. and German sample (Sachser et al., 2022). To determine if the Dutch questionnaire follows these cut off points we inspected the sensitivity, specificity and the Area Under the Curve (AUC), in which an AUC = .50–.60 is considered low accuracy, = .60–.80 as good accuracy, >.80 good accuracy and >.90 excellent accuracy.

3. Results

3.1. Confirmative factor analyses

We tested the second order four-factor structure of PTSD as defined in the DSM-5 model, consisting of the factors: intrusions (five items), avoidance (two items), negative alterations in cognitions and mood (NACM, seven items) and hyperarousal (six items). We used confirmatory factor analysis to examine the validity and this factor structure showed a good fit for the selfreport as indicated by the following fit indices: χ²(164) = 393, p < .001, RMSEA .049 (90% CI: .04–.06), SRMR = .039, CFI = .95, TLI = .94. For the caregiver report we found an acceptable fit indicated by χ²(164) = 672, p < .001, RMSEA .069 (90% CI: .06–.07), SRMR = .050, CFI = .90, TLI = .89. The standardized loadings are shown in Figures 2 and 3 (selfreport and caregiver report respectively). Almost all factor loadings are above 0.40, which gives evidence that the items correspond with the hypothesized PTSD construct. Notably, in both figures it is visible that question 8 shows a low factor loading (.26 in the selfreport and .32 in the caregiver report). In the caregiver report, question 16 had a value of .35. Furthermore, the four factors showed moderate to strong correlations with each other (range = .43–.70) (Table 5).

Figure 2.

Second ordel model of selfreport KJTS, with standarized loadings.

Note: PTSD = Post Traumatic Stress Disorder; NACM = Negative Alterations in Cognitions and Mood.

Figure 3.

Second ordel model of caregiver report KJTS, with standarized loadings.

Note: PTSD = Post Traumatic Stress Disorder; NACM = Negative Alterations in Cognitions and Mood.

Table 5.

Correlations of underlying factors of the four-factor DSM-5 model for PTSD.

	Intrusions	Avoidance	NACM	Hyperarousal
KJTS selfreport (n = 587)
Intrusions	1
Avoidance	.48	1
NACM	.68	.53	1
Hyperarousal	.58	.43	.70	1
KJTS caregiver report (N = 658)
Intrusions	1
Avoidance	.66	1
NACM	.59	.53	1
Hyperarousal	.55	.50	.70	1

Note: NACM = Negative alterations in cognitions and mood.

3.1.1. Internal consistency

We found an excellent reliability for both selfreport, α = .92 and caregiver report, α = .92. In Table 6 the internal consistency of the different criteria are displayed. Notably in the selfreport the Avoidance criterium (C) shows a questionable reliability. We calculated an Intraclass correlation coefficient (ICC) to determine agreement between the selfreport of the children and adolescents and their caregivers (N = 229) with an average age of 13.34 years (SD = 2.83, range = 7–21). An ICC value of .44 was found (95% CI: .33–.54), which indicates a poor agreement. Classifying the sample in three different age group shows that adolescents >16 years and their caregivers have a moderate agreement (.56, 95% CI: .34–.72) compared to the younger children and their caregivers, who show a very poor agreement. In Table 7 different age ranges are presented and their ICC values.

Table 6.

Reliabilities (Cronbach’s α) of the KJTS subscales for selfreport and caregiver report.

	α
KJTS 7–18 self total score (n = 587)	.92
Criterion B: Intrusions	.86
Criterion C: Avoidance	.68
Criterion D: NACM	.86
Criterion E: Hyperarousal	.72
KJTS 7–18 Caregiver total score (N = 658)	.92
Criterion B: Intrusions	.85
Criterion C: Avoidance	.79
Criterion D: NACM	.88
Criterion E: Hyperarousal	.75

Note: NACM = Negative alterations in cognitions and mood.

Table 7.

Intraclass correlation coefficients after segregation by age of the child.

	ICC	95% CI
7–11 years (N = 59)	.35	.11–.55
12–15 years (N = 115)	.38	.21–.53
>16 years (N = 55)	.56	.34–. 72

Note: ICC = intraclass correlation coefficient. CI = confidence interval.

3.2. Convergent and divergent validity pattern

The convergent-discriminant validity pattern has been analysed through bivariate analyses of the KJTS, CBCL and YSR. DSM-5 domains and the total domains on the CBCL en YSR are used, with a main focus on the bivariate outcomes between the selfreports (KJTS self and YSR) and the outcomes of the caregiver reports (KJTS caregiver and CBCL). As expected, the pattern shows a medium to strong correlation with internalization problems, such as anxiety and affective problems (r = .48–.72) and low to medium correlation with externalizing symptoms (r = .25–.37). Selfreports and caregiver reports show overall stronger correlation between themselves, whereas crossovers between selfreport and caregiver reports show overall lower correlations (shown in grey). All correlations are delineated in Table 8.

Table 8.

Bivariate outcomes (r) of trauma symptoms and internalizing and externalizing problems.

Measures	KJTS 7+ Selfreport		KJTS 7+ Caregiver report
	r	n	r	n
Selfreport YSR
YSR TOT	.68*	175	.45*	245
YSR INT	.72*	175	.44*	245
YSR EXT	.37*	175	.27*	245
DSM-5 scales
Affective problems	.63*	175	.42*	245
Anxiety	.57*	175	.39*	245
Somatic problems	.49*	175	.35*	239
ADHD	.31*	175	.25*	245
ODD	.21*	175	.17*	245
CD	.30*	175	.20*	245
Caregiver report CBCL
CBCL TOT	.28*	245	.57*	455
CBCL INT	.35*	245	.60*	455
CBCL EXT	.10*	245	.36*	455
DSM-5 scales
Affective problems	.44*	245	.60*	455
Anxiety	.28*	245	.48*	455
Somatic problems	.25*	239	.37*	445
ADHD	.11	245	.30*	455
ODD	.00	245	.23*	455
CD	.03	245	.25*	455

Note: CBCL = Child Behaviour Checklist, YSR = Youth Self Report, TOT = total score, INT = total score on internalization problems, EXT = total score on externalization problems, DSM-5 = Diagnostic and Statistical Manual of Mental Disorders Fifth edition, ADHD = Attention Deficit/Hyperactivity Disorder, ODD = Oppositional Defiant Disorder, CD = Conduct Disorder, KJTS = Kind en Jeugd Trauma Screener.

*p < .01.

3.3. Sensitivity analysis

The ROC-curve analysis of the KJTS selfreport validated against the CAPS-CA-5 diagnostic interview, has proven a good accuracy (AUC = .81; n = 106). Inspecting sensitivity and specificity a cut-off of <15 for non-clinically elevated symptoms (sensitivity = 95% and specificity = 69%); and a cut-off of  ≥ 21 points to be a sensitive screening cut-off for elevated trauma-related distress (sensitivity = 93% and specificity = 65%); and a cut-off of ≥ 25 points for a high trauma-related distress and probability on a PTSD diagnosis (sensitivity = 89% and specificity = 47%). The sensitivity and specificity of the cut-off scores are in line with the outcomes in CATS-2 study of Sachser et al. (2022).

4. Discussion

This study demonstrates the psychometric accuracy of the KJTS in a Dutch clinical population. Confirmatory factor analysis shows that our a priori hypothesis of the DSM-5 four-factor structure of PTSD has a good fit for the selfreports and an acceptable fit for the caregiver reports. This analysis gives evidence that the KJTS reflects adequately the dimensionality of PTSD as described in the latest version of the DSM (DSM-5; American Psychiatric Association, 2013). Selfreports show a better fit than caregiver reports, possibly because selfreports capture a more accurate understanding of the internal thoughts and feelings. Caregivers are often biased by their own perceptions, and it is a challenge for them to estimate what is going on internally in their child. As a result the child’s behaviour can be misinterpreted (Skar et al., 2021). While we argue that there is a good and acceptable fit, a point of attention is that both selfreport and caregiver report show respectively good and acceptable fits on all indices, except on the Tucker-Lewis Index (TLI) where they both come short of .01. CFI and TLI are correlated and TLI will always be smaller than CFI if CFI is <1. Nonetheless we reason that the self-report is a good fit and the caregiver report is an acceptable fit, since there is much discussion on the exact cut-off points, and it has been argued that the cut-off points are not set in stone and that it is more important to see how far off or close the indices are to −1 or 1 (West et al., 2023). Examination into the pattern of factor loadings showed that all four domains in both selfreport and caregiver report have a strong influence on the construct of PTSD, which supports the fit of the KJTS. Upon closer inspection of the items and their loadings, it is notable that question 8 ‘inability to remember an important aspect of the traumatic event(s)’ (i.e. psychogenic amnesia) loads relatively poor on its intended domain ‘NACM’, in both selfreport and caregiver report. This finding is consistent with the findings in several prior factor analytical studies of PTSS (King et al., 1998; Miller et al., 2013; Nilsson et al., 2021; Palmieri et al., 2007; Simms et al., 2002). However, previous research on the relationship between dissociation and PTSD shows that psychogenic amnesia was more often endorsed by individuals with a higher level of symptom severity (e.g. marked elevations in depersonalization, derealization, and flashbacks). This gives rise to the question if psychogenic amnesia might be an indicator of a distinct subtype or subgroup within PTSD (Miller et al., 2013). Therefore, it has been argued that it would be advantageous to drop this item from the core symptoms of the disorder and redefining it as a marker of a dissociative subtype (Miller et al., 2013). Current findings support the opening of this discussion on the classification of PTSD, also in children and adolescents.

Similar to previous findings, our study demonstrates that both selfreport and caregiver report versions have excellent internal consistency across all items (Alberici et al., 2018; Ford-Paz et al., 2019; Nilsson et al., 2021; Sachser et al., 2017; Usama et al., 2021). Exploring the internal consistency of the isolated domains indicates that the domains mostly exhibit a good internal consistency, compared to excellent in the total questionnaire. Especially avoidance presents a lower internal consistency, possibly due to the two scale domain, for which it is common to show low-reliability scores (Eisinga et al., 2013). The lower internal consistencies of the isolated domains support the construct of PTSD, where all four domains are necessary for the classification according to the DSM-5. Nevertheless, it is not necessary to meet all 20 symptoms, due to the diversity of expression of symptoms. Additionally, the correlation of avoidance with the three other clusters is relatively low. It can be argued that when avoidance is severely present, other symptoms might be suppressed or not recognized. This is seen in a group of adults with high avoidance rates and moderate outcome on PTSS (Barbieri et al., 2021). The four factors combined give an overall score that can be used in clinical care for assessing the possible presence of PTSD. It is important that this overall score is meant for screening purposes, and always need further exploration before classifying PTSD.

Comparing caregiver and selfreports, we have found a poor caregiver-child agreement (Sachser et al., 2017; Verlinden et al., 2014) indicating that there is a discrepancy between what the child reports and their caregiver. This discrepancy will be discussed more in depth below. Lastly, the presumed convergent-divergent pattern has been confirmed with a high correlation of the KJTS and internalizing problems and low correlation of the KJTS with externalizing problems. The current study is the first to confirm satisfactory reliability and validity in Dutch clinical samples of children and adolescents and is therefore a useful instrument for clinicians and researchers.

4.1. Scoring of the KJTS

The current cut-off scores, < 15 for non-clinically elevated symptoms, 15–20 for moderate trauma-related distress,  ≥ 21 as the sensitive screening cut-off for elevated trauma-related distress and  ≥ 25 for a high trauma-related distress and probability on a PTSD classification, are balanced between sensitivity (identifying true positives) and specificity (avoiding false positives) for the clinical child- and adolescent mental healthcare. Determining a cut-off score often involves to a certain degree a subjective judgment. The appropriate score may vary depending on the context, the purpose of the assessment, and the specific population being evaluated. Adjusting the cut-off may improve one measure, such as the specificity, at the expense of the other, the sensitivity. Since we do not want to miss potential PTSD, it is important to have a more sensitive test than specific test. Therefore we tolerate the relatively high specificity of 47% for the 89% sensitivity in the clinical elevated group (sum score ≥ 25). Moreover, PTSS has an extensive overlap with other symptoms of psychiatric disorders, such irritability, feelings of disconnection and sleeping and concentration problems (Brand et al., 2018; Rodríguez & Villegas, 2022) and PTSS are often overlooked or misjudged in mental healthcare (Kildahl et al., 2019; Lecrubier, 2004). With the chosen sensitivity, it is likely not to miss PTSS while doing a screening assessment. It remains a possibility that children show a high sum score on the KJTS, due to comorbid problems and not solely explained by PTSS. To adhere to this problem, clinicians have the opportunity to use a more dimensional scoring in addition to the sum score. In this scoring, the symptom scores are divided over the four domains of PTSD. A clinician can use this information to make a thorough decision on further diagnostic instruments and treatment.

4.2. (Dis)agreements between selfreports and caregiver reports

Caregiver reports are valuable in assessing a child's mental health issues or disorders as they can provide information about multiple aspects of the child's mental well-being. In our sample, there is in generally a low caregiver-child agreement on the PTSD symptoms, with a very low caregiver-child agreement in younger children (age 7–15 years) compared to the adolescent group (age 16+). Notably, we found a difference between amount of PTEs that are reported in the selfreport and the caregiver reports, 57% of the children report more than four PTEs, against 26% of the caregivers. This is an eminent finding, considering that the amount of events that have been present in the life of the child play an important role in the development of trauma-related symptoms; an increase in number or severity of traumatic life events also tends to enlarges the risk of developing PTSS and the risk of PTSD in adulthood (Ogle et al., 2014; Steine et al., 2017). Moreover, a low concordance between the child and their caregiver on the traumatic event(s) has been found related to higher levels of trauma-related symptoms in the child (Ceballo et al., 2001; Oransky et al., 2013) and lower self-esteem and problem-solving abilities (Howard et al., 1999). In our sample, over half of the children report more than four PTEs whereas a quarter of the caregiver reports more than four PTEs on the checklist. This discrepancy between selfreport and caregiver report is an alarming finding, considering that caregivers might not be aware of the PTEs of their children and do not recognize symptoms in their children as trauma-related symptoms. Using the KJTS can help clinicians to indicate these differences prior to, or during, treatment, to intervene with an individualized approach such as caregiver psycho-education. The low agreements between child and their caregiver are also found in the study of Sachser et al. (2017, 2022) and in the study of Verlinden and colleagues on the Dutch DSM-IV traumascreener (CRIES-13, 2014). However, an earlier study in 2007 on post-traumatic stress reactions in children, showed generally good caregiver-child agreement when the child was exposed to more clear or chronic stressors, such as bereavement, cancer and other chronic illnesses (Meiser-Stedman et al., 2007). A tenable argument is that obvious stressors such as bereavement and chronic illness may provide more opportunities for caregivers to notice distress and post-traumatic stress reactions in their children. In contrast, in case of a single or less obvious traumatic event, caregivers may be less aware of the possible psychological damage and distress in their child. Additionally, it is emphasized that some types of traumatic events, such as community violence and sexual abuse, may have happened without knowledge of the caregiver(s), which makes it more difficult to recognize post-traumatic stress reactions in their child (Skar et al., 2021). Regarding the present low caregiver-child agreements, it is of great importance to asses both child and caregiver and zoom in on the individual child and caregiver reports regarding life events and symptoms, especially in the younger age group where symptoms differ between the caregiver and child. Another future implication arising from this finding, is that creating awareness in both caregiver and the child on potential traumatic events and subsequent symptoms, might be precedence. This will help the caregiver to recognize and acknowledge the events and subsequential impact, since it has been found very challenging for caregivers to notice these events (Marsac et al., 2013). Additionally the manifestation of PTSD is very divers, and bringing awareness might affect the reports on symptoms. A further study should focus on the relation between the knowledge on PTE, PTSD and the reported outcomes to have more in-depth insight. Another interesting caregiver-child finding is the under-representation of fathers (15% of all caregiver reports). Previous studies found differences between fathers’ and mothers’ reports on anxiety symptoms, which can lead to different outcomes (Davé et al., 2008; Jansen et al., 2017; Krain & Kendall, 2000; Treutler & Epkins, 2003). It has been argued that relying on only one parent or caregiver, especially in disorders characterized by covert symptoms, can result in different conclusions (Jansen et al., 2017). Furthermore, Treutler and Epkins (2003) emphasize the significant influence of mothers’ and fathers’ psychological symptoms and other parent–child relationship variables on the rating of their child’s externalizing and internalizing behaviour. It is therefore clinically relevant to have both caregivers report on the symptoms of their child and to stimulate this by clinicians and the mental health facility.

4.3. Limitations

The current study knows its limitations. Firstly, data has been collected from one institution, Levvel youth mental healthcare in Amsterdam, which can affect the diversity of the sample. However, children from multiple surrounding cities and areas are included, with a variety in age, gender, place of birth and psychiatric problems. Therefore, we surmise that the results can be generalized towards the Dutch youth psychiatry population. A second limitation is that test-retest reliability was not evaluated in the current study. In the future the screener has to be measured in a test-retest design during a stable phase and also during trauma therapy to assess the sensitivity to change. Furthermore, the sensitivity/specificity analyses are assessed with the CAPS-CA-5. This semi structured clinical interview is not (yet) validated in the Dutch population, which can affect the reliability of the analyses. Nonetheless, the CAPS-CA-5 is seen as the golden standard internationally and the predecessor, the CAPS-CA DSM-IV, showed good psychometric proportions (Diehle et al., 2015). Thus we decided to use the outcomes of the CAPS-CA 5 for the purposes of the sensitivity-specificity analyses. Lastly, the PTEs checklist is – due to logistic reasons – not entirely identical to the PTEs of the international group. Comparing and analysing these PTEs further in international research is therefore a challenge. It becomes more and more clear that the cumulative effect of PTEs affect the child, and their development into later adulthood (da Silva et al., 2024; Ogle et al., 2014; Steine et al., 2017). Subsequently more thorough research combining international knowledge and strength is necessary towards the recorded PTEs and the prevalence and impact on children and adolescents. To adhere to this dispute, we readjusted in 2023 the life event checklist, making it comparable with the international research group. An important example of an adaptation is the addition of bullying in real life and cyber bullying.

5. Conclusion

The KJTS is a reliable and valid questionnaire for the assessment of the risk on PTSD in children and adolescents according to the most commonly used classification model in the Netherlands, the DSM-5. The questionnaire is available free of charge and easy to use for children and adolescent mental healthcare professionals without extra training. It assists the professional to estimate the prevalence and severity of potentially traumatic events and PTSS. In the future it is necessary to replicate this study to be sure of the results, and to implement a test-retest design, in order to measure the stability in a baseline situation as well as the sensitivity for change. This sensitivity for change will help us monitor trauma focused treatments and help us to target the symptoms more effectively.

Funding Statement

This study has been financially supported by the Dutch Government: Rijksmiddelen t.b.v. academisch component kinder- en jeugd psychiatrie.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author, LHK, upon reasonable request.