Development and validation of the Diagnostic Interview Adjustment Disorder (DIAD)

Abstract

Adjustment disorders (ADs) are under‐researched due to the absence of a reliable and valid diagnostic tool. This paper describes the development and content/construct validation of a fully structured interview for the diagnosis of AD, the Diagnostic Interview Adjustment Disorder (DIAD). We developed the DIAD by partly adjusting and operationalizing DSM‐IV criteria. Eleven experts were consulted on the content of the DIAD. In addition, the DIAD was administered by trained lay interviewers to a representative sample of disability claimants (n = 323). To assess construct validity of the DIAD, we explored the associations between the AD classification by the DIAD and summary scores of the Kessler Psychological Distress 10‐item Scale (K10) and the World Health Organization Disability Assessment Schedule (WHODAS) by linear regression. Expert agreement on content of the DIAD was moderate to good. The prevalence of AD using the DIAD with revised criteria for the diagnosis AD was 7.4%. The associations of AD by the DIAD with average sum scores on the K10 and the WHODAS supported construct validity of the DIAD. The results provide a first indication that the DIAD is a valid instrument to diagnose AD. Further studies on reliability and on other aspects of validity are needed. Copyright © 2014 John Wiley & Sons, Ltd.

Introduction

The term adjustment disorder (AD) is used to describe a condition where an individual reacts to a stressful event with disproportionate symptoms and behaviors. AD is considered to be a common mental health problem in the general population, in primary and in secondary care (Casey, 2009). Although usually believed to be mild and self‐limiting, AD is associated with long‐term sickness absence and disability (van der Klink et al., 2003). In several countries, stress‐related disorders are one of the most commonly reported types of work‐related illness (Health and Safety Executive, 2011; Knowledge Center UWV, 2007; National Institute for Occupational Safety and Health, 2011). In the Netherlands, stress‐related disorders including AD are the second most important psychiatric diagnosis on certificates of benefit claimants, with a prevalence of 6.7% after one year, and of 4.8% after two years of sickness absence (Knowledge Center UWV, 2007).

The diagnostic instrument most commonly used in psychiatric epidemiological research, the Composite International Diagnostic Interview (CIDI), lacks a section dealing with AD (Kessler and Üstün, 2004). None of the large‐scale epidemiological surveys on mental health carried out in the United States and in Europe included AD for consideration (Kessler et al., 2005; Bebbington et al., 2000; Vollebergh et al., 2001; Wittchen and Jacobi, 2005). Only the Outcome of Depression International Network (ODIN) study, a general population survey carried out in five European countries, assessed AD (Casey et al., 2006). In that study, the prevalence of AD was found to be extremely low: 0.0% to 1.0%. Other studies that also incorporated the diagnosis of AD, used a variety of diagnostic methods and showed a varying prevalence of AD (Casey, 2009). As a result, reliable information on the prevalence and course of AD is scarce, and few strategies for the treatment of AD are evidence‐based (van der Klink et al., 2003).

Criteria for the diagnosis of AD are described in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (American Psychiatric Association, 1994), see Table 1.

Table 1.

Criteria for adjustment disorder with diagnosis algorithm, specified according to the DSM‐IV and adjusted/operationalized for use in the present study

nr	Criteria	DSM‐IV	Adjusted/operationalized
1	Stressor	identifiable stressor(s)	cluster of identifiable stressors in recall period of three years
2	First time limit	occurring within three months of the onset of stressor(s)	symptoms within three months after onset of stressor cluster
3	Distress	clinically significant as evidenced by marked distress that is in excess of what would be expected	4DSQ distress scale scoring > 10
4	Impairment	clinically significant as evidenced by significant impairment in social or occupational (academic) functioning	SDS impairment scale scoring ≥ 4 in at least two domains
5	Second time limit	once the stressor (or its consequences) has terminated, the symptoms do not persist for more than an additional six months	not used
6	DSM‐IV Axis I/II	the stress‐related disturbance does not meet the criteria for another specific Axis I disorder and is not merely an exacerbation of a pre‐existing Axis I or Axis II disorder	not used
7	Bereavement	symptoms do not represent bereavement	symptoms due to bereavement or loss of health due to serious illness/injury as single stressor need to be present for longer than 12 months.
Diagnosis algorithm		1 AND 2 AND (3 OR 4) AND 5 AND 6 AND 7	1 AND 2 AND 3 AND 4 AND 7

These criteria are generally considered to be vague and ill‐defined (Casey et al., 2001). In recent years, the non‐specific classification of AD in the DSM‐IV has been under dispute (Baumeister and Kufner, 2009; Baumeister et al., 2009; Casey and Bailey, 2011; Laugharne et al., 2008). Some critics argue that the concept AD medicalizes problems of ordinary life, while others put forward that a rigid “cook‐book” application of diagnostic criteria may result in an over diagnosis of other psychiatric disorders at the cost of AD (Casey et al., 2006). More specifically, the critique of the current DSM‐IV conceptualization of AD concentrates upon the inadequate definition of clinical significance, the failure to distinguish AD from other Axis I disorders and the neglect of contextual factors accounting for excess symptoms of AD (Baumeister et al., 2009). Therefore, Baumeister et al. (2009) recommended to eliminate the criterion that requires the absence of another DSM‐IV disorder, to define clinical significance with the requirement that both distress and impairment are present and to extend the bereavement exclusion criterion to other severe or uncommon stressful events (Baumeister et al., 2009). These recommendations are still worth considering, since the proposed classification of AD in the upcoming Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (American Psychiatric Association, 2011) is very similar to the DSM‐IV.

Recently, a questionnaire for the assessment of AD was developed and validated (Einsle et al., 2010). However, this instrument is based on a new diagnostic proposal, that places AD in a spectrum of stress‐response syndromes, along with post‐traumatic stress disorder (Maercker et al., 2007). As this questionnaire is not compatible with DSM‐criteria, a valid diagnostic instrument that enables lay interviewers to assess AD based on DSM‐IV criteria is still missing. The present study is an attempt to make up for this deficiency by developing and validating a fully structured interview to diagnose AD, the Diagnostic Interview Adjustment Disorder (DIAD), that can be administered by lay interviewers, based on adjusted DSM‐IV criteria as recommended by Baumeister et al. (2009). We aimed to assess the content and the construct validity of the DIAD. Regarding construct validity, distress and impairment are defined as core symptoms of AD. Therefore, it can be expected that the diagnosis AD is associated with these symptoms. Therefore, we hypothesized that persons with AD have higher levels of distress and impairments than persons without AD.

Methods

Setting and participants

The present study is part of a cohort study on prognostic factors of long‐term disability due to mental disorders, Predicting Disability (PREDIS) (Cornelius et al., 2013). Participants eligible for PREDIS were recruited using registry data from the local office of the Social Security Institute (SSI) in the city of Groningen, servicing Groningen and Drenthe, two northern provinces of the Netherlands. Recruitment started at October 1, 2008 and ended at December 31, 2009. Included were persons claiming disability benefit after two years of sickness absence due to any medical condition, whether somatic or mental. The SSI uses the International Classification of Diseases, 10th Revision (ICD‐10) to certify diagnoses as cause of disability. The Medical Ethics committee of the University Medical Center Groningen (UMCG) approved recruitment, consent and field procedures. Out of a total of 1544 eligible long‐term sick listed workers, 375 persons consented to participate (response rate = 24.3%), see Figure 1 for a flow chart of participants.

Figure 1.

Flowchart of participants.

At the first measurement, i.e. after two years of sickness absence, respondents were sent a questionnaire on demographics, general and mental health, alcohol use, functioning, health care use, coping behavior and social support. After they completed and returned the questionnaire, respondents were interviewed at home by lay interviewers using the CIDI, supplemented by the DIAD. The median time between completing the questionnaire and the CIDI/DIAD was four weeks (standard deviation [SD] five weeks). For the present study, we included only those participants from whom we could obtain complete interview data, both from the CIDI and the DIAD. As a result, the study sample consisted of 323 CIDI/DIAD completers.

To assess generalizability, we compared PREDIS responders (n = 375) with non‐responders (n = 1169) as to age, gender and ICD‐10 diagnosis on SSI certificates as cause of disability. We found no significant differences between responders and non‐responders as to gender (p = 0.850; χ ² = 0.036) and ICD‐10 classifications of somatic and mental disorder as cause of disability (p = 0.682; χ ² = 1.500). As to age, we found responders to be significantly older than non‐responders (p < 0.001; χ ² = 60.022). Age categories 45–54 years and 55–65 years are over‐presented in the study sample. We also compared the PREDIS cohort with a large national population (n = 56,267) of all persons claiming disability benefit in the years 2006–2007 (Knowledge Center UWV, 2007). We found the sample not to differ significantly from this national population as to prevalence of ICD‐10 somatic (p = 0.876; χ² = 1.214) and ICD‐10 mental disorders, i.e. mood, anxiety and stress‐related disorders (p = 0.344; χ ² = 7.870), certified by the SSI as primary cause of disability.

Development

The DIAD was developed by two (BC, JvdK) of the authors as a structured interview to diagnose AD based on DSM‐IV criteria and adjusted following the recommendations of Baumeister et al. (2009). Some of these adjusted criteria needed further operationalization. The result was a set of adjusted and operationalized criteria for the diagnosis AD, dealing with recall period, stressor(s), time relations between stressor and complaints, clinical significance, co‐occurrence with other DSM‐IV disorders and bereavement, see Table 1.

Recall period

The DSM‐IV does not set a recall period for stressors to have occurred. Since the DIAD was to be administered to persons claiming disability benefit after two years of sickness absence, we chose a recall period to capture any stressor that may be related to the onset of sickness absence. Therefore, we set the period of recall at three years.

First time limit criterion

We expected respondents to probably report more than one stressful life event, i.e. a cluster of stressors, to have occurred in this recall period. Each stressor within a cluster may have different dates of onset and termination. We considered the first time limit criterion to be met, if the onset of symptoms occurred within three months of the onset of at least one of the stressors within the cluster.

Clinical significance criterion

We revised the DSM‐IV clinical significance criterion and followed Baumeister et al. (2009), requiring that both marked distress and significant impairment are present. To operationalize distress and impairment, the DIAD incorporates two reliable and valid scales: the distress subscale of the Four‐dimensional Symptom Questionnaire (4DSQ) (Terluin et al., 2006) and the Sheehan Disability Scale (SDS) (Leon et al., 1997). Although these two scales are commonly employed as paper–pencil questionnaires, we used them as part of the DIAD. In accordance with existing scoring rules, we defined a sum score > 10 on the 4DSQ subscale distress as marked distress and a SDS sum score ≥ 4 in at least two domains of the SDS as significant impairment.

Second time limit criterion

The DIAD does not assess the second time limit criterion stated in the DSM‐IV, i.e. the criterion that the symptoms must have resolved within six months once the stressor has terminated.

DSM‐IV exclusion criterion

By definition of DSM‐IV, the diagnosis AD cannot be made if the condition meets the criteria of an Axis I mood or anxiety disorder, or is merely an exacerbation of a pre‐existing Axis I or Axis II disorder. We deleted this exclusion criterion, following the recommendation of Baumeister et al. (2009).

Bereavement exclusion criterion

Normal sadness due to bereavement after the death of a loved one, or similar types of loss should be excluded from the diagnosis AD, while pathological or dysfunctional reactions should be included. We defined similar types of loss as loss of health due to a serious illness or injury. According to the proposed classification of AD in the DSM‐V, reactions due to bereavement (or similar types of loss) are pathological or dysfunctional when they persist for more than 12 months after the event. Therefore, we adjusted the DSM‐IV bereavement criterion to only include persons with symptoms exclusively representing bereavement after the death of a loved one or similar types of loss and lasting longer than 12 months.

AD subtypes

We did not expand the DIAD with questions aiming to subtype AD with depressed mood, anxiety, disturbance of conduct or combinations thereof.

Content of the DIAD

We used the DIAD to diagnose AD in the study sample. The DIAD contains 29 questions, see Table 2.

Table 2.

Content of the Diagnostic Interview Adjustment Disorder (DIAD)

Element	item	Assessment of
Stressor	1–3	identification of stressor(s)
	4–6	onset and duration of stressor(s)
distress	7–22	distress domain (from 4DSQ a)
	23	onset of distress
distress‐stressor	24	relation distress – stressor(s)
impairment	25–29	impairment (SDS b)

^aFour‐dimensional Symptom Questionnaire (Terluin et al., 2006).

^bSheehan Disability Scale (Leon et al., 1997).

The DIAD starts with three questions to identify and specify stressful live events that have occurred in the past three years. Respondents were asked to select one or more stressor(s) from a list of stressors, followed by three questions assessing the date at which the stressor(s) occurred for the first time, whether the stressor was still present at the moment of the interview and, if not, when the stressor ceased to exist. The DIAD then assesses levels of distress caused by the reported stressor(s) with 16 questions, i.e. the distress domain of the 4DSQ. The DIAD then asks when distress symptoms have started and whether these symptoms are a reaction to the stressful events mentioned earlier on in the interview. Finally, the last five questions of the DIAD focus on levels of impairment as a consequence of the reported distress symptoms, using the SDS. We have added a full transcript of the DIAD as an appendix to this paper.

Content validation

We developed the DIAD within the author group until we felt it to have sufficient face and content validity to be used in the study. After the study started using this initial version of the DIAD, we sought expert opinions on our choices and decisions we had already made in the initial development of the DIAD. This means that our use of the DIAD in the study population and our asking the opinion of selected experts was a parallel process. To assess whether the DIAD captured all essential aspects of AD, we asked 11 experts in relevant fields of psychiatry, psychiatric epidemiology, primary, occupational and insurance health care, and instrument development (see acknowledgments) to review a written transcript of the DIAD (see Appendix) and to complete a 17‐item questionnaire (see Table 3).

Table 3.

Expert opinion (n = 11) on content validity of the DIADa for the diagnosis adjustment disorder (AD)

item	What is your opinion on:	Disagree	Neutral	Agree
1	our decision to set the recall period duration at three years?	1 (9.1)	5 (45.5)	5 (45.5)
2	our assumption that respondents are unable to attribute complaints to separate stressors with overlapping time frames?	5 (45.5)	2 (18.2)	4 (36.4)
3	our decision to consider stressors with overlapping time frames as a single problem cluster?	1 (9.1)	1 (9.1)	9 (81.8)
4	our assumption that respondents are able to attribute complaints to a cluster of stressors xwith overlapping time frames?	1 (9.1)	1 (9.1)	9 (81.8)
5	our decision to have the DIAD assess xdistress complaints only?	3 (27.3)	4 (36.4)	4 (36.4)
6	our choice for the Distress scale of the 4DSQ to assess distress complaints?	1 (9.1)	0 (0.0)	10 (90.1)
7	our decision not to have the DIAD assess depressed mood?	4 (36.4)	4 (36.4)	3 (27.3)
8	our decision not to have the DIAD assess anxiety?	5 (45.5)	3 (27.3)	3 (27.3)
	our decision not to have the DIAD assess disturbance of conduct?	2 (18.2)	5 (45.5)	4 (36.4)
10	our decision not to have the DIAD assess DSM‐IV Axis II disorders?	1 (9.1)	0 (0.0)	10 (90.1)
11	our decision that the first time criterion is met, if at least one stressor started within three months preceding the onset of symptoms?	1 (9.1)	0 (0.0)	10 (90.1)
12	our decision not to have the DIAD assess whether the second time criterion is met?	4 (36.4)	1 (9.1)	6 (54.5)
13	our assumption that respondents are well able to self‐assess whether their complaints are a reaction to the stressor they experienced?	2 (18.2)	3 (27.3)	6 (54.5)
14	our choice for the Sheehan Disability Scale to assess impairments?	0 (0.0)	5 (45.5)	6 (54.5)
15	our assumption that, in this specific population of persons with long term disability, lay interviewers are well able to assess whether the distress is in excess of what would be expected from exposure to the stressor?	7 (63.6)	3 (27.3)	1 (9.1)
16	the position that the DIAD covers essential aspects of the DSM‐IV diagnosis adjustment disorder?	0 (0.0)	1 (9.1)	10 (90.1)
17	the position that the DIAD as a supplement to the CIDI has added value for the assessment of adjustment disorder in psychiatric epidemiologic research?	1 (9.1)	1 (9.1)	9 (81.8)

^aDiagnostic Interview Adjustment Disorder.

Construct validation

Administration

We tested our hypotheses by administering the DIAD to the PREDIS cohort. Twelve lay interviewers were trained by certified trainers from the Dutch CIDI Training Center in Groningen, the Netherlands and by the first author. Respondents were interviewed face‐to‐face at their home. The DIAD was administered immediately after completion of the CIDI. Interviewing was laptop assisted. Quality of interviewing techniques was evaluated bimonthly in training sessions.

Measures

To assess distress and impairment, the questionnaire administered to PREDIS respondents included the Kessler Psychological Distress 10‐item Scale (K10) (Kessler et al., 2002) and the World Health Organization Disability Assessment Schedule version 2.0 (WHODAS 2.0) (Üstün et al., 2010).

The K10 consists of 10 items with each five response categories: “none of the time” (1), “a little of the time” (2), “some of the time” (3), “most of the time” (4) and “all of the time” (5). Sum scores range from 10 to 50. The K10 has strong psychometric properties and is widely used as screener for psychological distress (Kessler et al., 2002).

With 36 questions, the WHODAS 2.0 captures levels of functioning in six domains of life: Understanding and Communicating (six items), Getting around (five items), Self‐care (four items), Getting along with people (five items), Life activities (household activities: four items; work: four items) and Participation in society (eight items). Answering options are “none” (1), “mild” (2), “moderate” (3), “severe” (4) and “extreme/cannot do” (5). Sum scores run from 36 to 180. The WHODAS 2.0 has excellent psychometric properties (Buist‐Bouwman et al., 2008).

The CIDI was used to assess other 12‐month DSM‐IV disorders co‐occurring with the diagnosis AD. The CIDI is a laptop assisted fully‐structured interview to be administered by lay interviewers. The validity of the CIDI in assessing mental disorders is generally good, as compared with structured diagnostic interviews administered by clinicians (Haro et al., 2006).

Data analysis

For content validity, expert agreements greater than 80% were considered good, between 50% and 80% moderate, and lower than 50% poor (slightly adapted from Altman, 1991). DIAD data were merged from the different laptops used for the CIDI/DIAD interviews. With these data, we made the diagnosis AD post hoc with an algorithm, using the criteria presented in Table 1, dividing the study population in a group with AD and a group without AD. For construct validity, we calculated sum scores of the K10 and the WHODAS. To evaluate our hypotheses on the expected associations of the diagnosis AD with these sum scores, we performed simple linear regression analyses with the diagnosis AD as an independent variable and the sum scores of the K10 and the WHODAS as dependent variables. The standardized coefficients provided by linear regression represent how many standard deviations the scale scores differed, depending on whether AD is present or not. Calculation of the standardized coefficients enables ranking the effect of the presence of AD on the scores of scales. We used a confidence interval of 95% and a level of significance of 0.05. Data were statistically analysed with IBM SPSS version 19.0 for Windows.

Results

Content validity

The experts opinion on the content of the DIAD is presented in Table 3. Good agreement (more than 80%) was reached on items 3, 4, 6, 10, 11, 16 and 17. The experts were in moderate agreement (between 50% and 80%) on items 12, 13 and 14. Poor agreement (lower than 50%) was found on items 1, 2, 5, 7, 8, 9 and 15. Lowest agreement was obtained on item 15.

Description of study sample

Table 4 presents the prevalence of AD as measured with the DIAD, demographics (age, gender), DSM‐IV comorbidity and number as well as nature of reported stressors for the total study sample, and the distribution of these variables in two sub‐samples, one sample with AD (n = 24) and one sample without AD (n = 299).

Table 4.

Descriptive statistics (%) for the total sample, and for persons that fulfill (AD+) and not fulfill (AD−) the criteria for adjustment disorder (AD) based on the DIADa

	Total (n = 323)	AD + (n = 24)	AD– (n = 299)
Prevalence AD	7.4	100.0	0.0
Mean age	49.9	41.6	50.3
Gender (female)	50.2	58.3	49.5
12‐month DSM‐IV comorbidity b
any disorder	42.4	75.0	39.8
mood disorder	25.7	45.8	24.1
anxiety disorder	30.3	66.7	27.4
mood & anxiety disorder	12.2	41.7	12.0
Number of stressors
0	31.0	—c	33.4
1	22.9	8.3	24.1
2	11.8	20.8	11.0
>2	34.3	70.1	31.5
Nature of stressor
work	41.3	66.7	37.8
own illness	40.4	70.8	36.5
illness of other(s)	9.9	12.5	9.4
psychosocial	28.5	70.8	24.1

^aDiagnostic Interview Adjustment Disorder.

^bAssessed by the CIDI.

^cBy definition.

We found 24 respondents (7.4%) to meet all our criteria for AD. In both sub‐samples with and without AD, we found a high comorbidity of 12‐month mood and anxiety disorders: 45.8% and 66.7% respectively (with AD) and 24.1% and 27.4%, respectively (without AD). The prevalence of a mixed mood and anxiety disorder within the group diagnosed with AD was 41.7%, and in the group without AD 12%. More than 90% of respondents with AD reported more than one stressor to have occurred in the three year recall period. In the group without AD, multiple stressors were reported in 42% of cases. In both groups, stressors most often reported were those related to own illness, psychosocial factors and work. We found no respondents with bereavement or injury as single stressor while meeting all other criteria for AD (not shown in table). Using the DIAD, we classified two respondents with AD that reported sustained distress two years after they were diagnosed with a serious illness (not shown in table).

Construct validity

In Table 5, we present the results of linear regression with the K10 and the WHODAS scores as dependent variables to explore associations with the diagnosis AD (AD+), using the absence of the diagnosis AD (AD−) as reference category.

Table 5.

Associations of adjustment disorder (AD) based on the DIADa with the K10b and the WHODAS 2.0c sum scores for groups classified with AD (AD+) and without AD (AD−)

	Range	Mean ± SD		Unstandardized coefficient B (95% CI)	Standardized coefficient Beta	p‐Value
		AD + (n = 24)	AD– (n = 299)
K10	10–50	28.17 ± 6.18	20.91 ± 7.37	7.26 (4.14 to 10.37)	0.26	<0.001*
WHODAS 2.0
communication	6–30	14.96 ± 5.99	11.37 ± 5.08	3.59 (1.39 to 5.79)	0.18	0.001*
getting around	5–25	11.48 ± 4.65	11.33 ± 5.03	0.15 (–1.98 to 2.29)	0.01	0.888
self‐care	4–20	6.09 ± 2.76	5.53 ± 2.40	0.56 (–0.48 to 1.59)	0.06	0.289
getting along	5–25	11.35 ± 3.55	9.11 ± 3.76	2.23 (0.64 to 3.83)	0.16	0.006*
household	4–20	12.82 ± 4.68	10.39 ± 4.45	2.43 (0.48 to 4.38)	0.14	0.015*
work	4–20	13.86 ± 4.49	11.27 ± 4.91	2.59 (–0.10 to 5.28)	0.15	0.059
participation	8–40	24.39 ± 6.48	19.26 ± 6.04	5.13 (2.54 to 7.72)	0.22	<0.001*
total	36–180	94.08 ± 22.32	75.06 ± 22.27	19.02 (6.29 to 31.74)	0.23	0.004*

^aDiagnostic Interview Adjustment Disorder

^bKessler Psychological Distress Scale 10‐items.

^cWorld Health Organization Disability Assessment Scale version 2.0.

^* p < 0.05.

The unstandardized regression coefficients (B) shown in Table 5 represent the mean differences in all scores between groups with and without AD. For example, persons with AD scored 7.26 points higher on the K10 than persons without AD. We found AD associated with statistically significant higher scores on the K10 and the WHODAS subscales Communication, Getting along, Household activities and Participation, and the WHODAS Total. The differences we found in scores of other scales were not statistically significant, although in the expected direction.

Discussion

The experts we consulted on content validity of the DIAD were in moderate to good agreement on most items we used for the concept of AD. With regard to construct validity, our hypothesis was confirmed that persons diagnosed by the DIAD with AD score higher on levels of distress and impairment, than persons not diagnosed with AD.

Content validity

Good expert agreement

The experts were in good agreement on items 3, 4, 6, 10 and 11 and almost 90% of them felt that the DIAD covers essential aspects of the DSM‐IV diagnosis AD.

Moderate expert agreement

The experts were in moderate agreement on items 2, 12, 13 and 14. The expert opinion on item 2, i.e. our assumption that respondents would not be able to attribute any distress symptoms to a separate stressor, when in a certain period more than one stressor were present, was inconclusive. However, they strongly agreed (81.1%) with our decision to consider stressors with overlapping time frames as a single problem cluster (item 3) and our assumption that respondents are able to attribute complaints to such a cluster of stressors (item 4) and not to each stressor separately.

Item 12 deals with our elimination of the second time limit criterion, i.e. that the symptoms must have resolved within six months once the stressor has terminated. The possibility that our elimination of this DSM‐IV criterion resulted in false‐positive or false‐negative diagnosis of AD should be discussed. The first section of the DIAD asks whether a stressor has been present in the past three years and, if so, at what date it started, if it is still present and, if not, when it ended. The DIAD then asks about present state distress complaints. If distress is still present three possibilities exist – either the person has a chronic AD, or the person has developed a new condition or the diagnosis at the outset was not AD but some other disorder. If distress is absent two possibilities exist – either AD has resolved or some other disorder causing distress has resolved. In our opinion, therefore, the elimination of the second time limit was justified so as to avoid false positive or false negative diagnoses. Strict application of the second time criterion would imply that a diagnosis of present state AD is never possible and that the diagnosis AD can be made in retrospect only, when both stressor and symptoms no longer exist. In our view and in line with that of other authors (van der Klink and Terluin, 2005), application of this second time criterion makes the diagnosis AD clinically less relevant. Furthermore, we found that more than 70% of respondents with AD reported their own illness as one of multiple stressors. It is reasonable to assume that in this specific population of long‐term sick listed workers, the illness underlying the disability, is of a chronic nature with enduring consequences. This implies that most AD found in the study sample can be specified as chronic and that the deletion of the second time criterion had no effect on the prevalence of AD and our validity estimates.

With the moderate expert agreement on item 13, we are fairly confident that the reported symptoms were a reaction to the reported stressor.

Item 14 deals with our choice of the SDS to measure impairment. We included the SDS predominantly for practical reasons. The CIDI administered immediately prior to the DIAD, contained the SDS as well. Having the DIAD assessing impairment using yet another scale, would in our view have confused respondents, resulting in biased answers.

Poor expert agreement

We found the experts in poor agreement on five of the 14 items we used: 1, 5, 7, 8, 9 and 15. Item 1 deals with our choice to set the recall period at three years. This particular recall period was chosen to capture any stressor related to the onset of sickness absence, two years before the interview. As any other psychiatric diagnostic interview, the DIAD is an instrument based on self‐report. Due to the lengthy recall period, respondents may have been unable to reliably recollect dates of onset and termination of stressing circumstances, resulting in biased assessment of the first time limit criterion for the diagnosis AD. There is a very extensive body of knowledge on the relation between stress and memory. It shows that stressful experiences may produce intense, long‐lasting memories of the events themselves, while stress may also impair subsequent attention and memory and can even induce profound amnesia (Kim and Diamond, 2002). In general, with a probing sequence of age‐of‐onset questions, individuals are well able to recollect how old they were when certain events occurred or when certain symptoms began (Kessler et al., 2007; Knauper et al., 1999). However, reliable assessment of the AD time limit criterion requires precise recollection in terms of days or weeks, making age‐of‐onset questions useless. This potential recall bias may be two‐sided, because respondents may erroneously indicate a date too early or too late. This will therefore most likely not have influenced our estimate of the prevalence of AD, but will have underestimated the associations between the AD diagnosis and the other constructs in our construct validity study.

Item 5 deals with our decision to have the DIAD assess distress complaints only. Consistent with this, expert agreement on our decision not to assess subtypes of AD, i.e. depressed mood (item 7), anxiety (item 8) and disorder of conduct (item 9), were poor as well. We had several reasons for not assessing these subtypes. First, since we expected a relatively high prevalence of mental health problems in our study sample of long‐term sick listed workers, resulting in a lengthy CIDI interview time, and since the DIAD was to be administered after completion of the CIDI, it was important to balance interview burden for respondents and DIAD performance. Therefore, we limited the DIAD to assess key symptoms of distress and impairment only. Second, strictly speaking, these subtypes are not inclusive or exclusion criteria for the diagnosis of AD. Third, it is not yet certain how AD will be subtyped in the upcoming DSM‐V. Had we included assessment of DSM‐IV subtypes, the DIAD would possibly have soon been outdated.

Regarding item 15, as we expected, most experts (9.1%) felt that lay interviewers are not able to assess whether distress symptoms are in excess. This confirmed our decision earlier in the developing process to assess clinical significance with the distress scale of the 4DSQ, instead of having the interviewer assess clinical significance.

We did not specifically ask the experts opinion about our adoption of the recommendations by Baumeister et al. (2009). These recommendations are subject to a broader discussion (Baumeister and Kufner, 2009; Laugharne et al., 2008) about the classification of AD in the DSM‐IV and whether criteria for AD should be adjusted in the upcoming DSM‐V. The operationalization of the bereavement exclusion criterion in particular is difficult, since it requires a normative discussion about the threshold between normal and pathological reactions to stressing events. Persons with normal symptoms of distress and impairment due to bereavement or other uncommon/severe stressors, should be excluded from the diagnosis of AD, while those with pathological or dysfunctional symptoms should not. It seems reasonable to assume that our operationalization of the bereavement criterion, following both Baumeister et al. (2009) and the proposed classification of AD in the upcoming DSM‐V, excluded respondents with normal reactions to a stressing event.

Construct validity

Prevalence

The prevalence of AD using the revised criteria was 7.4%. That is much higher than the prevalence of 0.0% to 1.0% found in the ODIN study (Casey et al., 2006). The explanation for this large prevalence difference may be that in the present study mood and anxiety disorders are allowed to be comorbid with AD, while in the ODIN study using strict ICD‐10 criteria, they are not. This confirms the assumption of DSM‐IV critics that strict “cook‐book” application of all diagnostic criteria for AD leads to over‐diagnosis of mood disorders at the expense of AD (Casey et al., 2001; Baumeister et al., 2009; Taggart et al., 2006). These mood disorders may in fact be self‐limiting periods of low mood triggered by stressful events and be misdiagnosed as depression.

In the present study we used the DIAD in combination with the CIDI and found an AD prevalence of 7.4%. Therefore, we believe that the results of our validation study indicate that the DIAD is able to differentiate between AD and depression.

Stressors

As we expected, a vast majority of persons diagnosed with AD reported multiple stressors, mostly related to work, own illness and psychosocial factors. However, in the sub‐sample without AD, almost half of respondents also reported multiple stressful life events. A post hoc analysis of the study sample showed that 15.2% (n = 49) of respondents reported one or more stressors in the past three years, without meeting criteria for AD nor for any other lifetime DSM‐IV classification. This illustrates that some individuals react to stressors with clinically significant symptoms, while others do not.

K10 and WHODAS

On a scale of 10 to 50, we found the K10 score to differ seven points (21 versus 28) between persons without and with AD, respectively. On a scale of 36 to 180, the sum score of the total WHODAS 2.0 was found to differ 19 points (75 versus 94). Meaningful score differences should have not only statistical significance, but clinical relevance as well. To our knowledge, the smallest relevant difference in K10 score is not known. However, based on K10 validity studies (Donker et al., 2003; Furukawa et al., 2003), we believe a seven point difference in K10 score to be clinically meaningful. As to the clinical significance of the difference we found in WHODAS 2.0 score, also for this questionnaire a meaningful cutoff value is not known. In a group of persons with depression before and after rehabilitation, a decrease of 13 points in WHODAS mean total score was found (Pösl et al., 2007). Therefore, in our view, the difference of 19 points we found in WHODAS 2.0 total sum score between persons with and without AD, is clinically significant as well.

Limitations

Some limitations of this study must be taken into account. First, the present study describes the development of a new instrument and is a first effort to validate it. We did not yet assess the reproducibility of the DIAD. Therefore, pending further studies on interrater and intrarater reliability and on other aspects of validity, the DIAD can only be used with prudence. Second, it has been pointed out by others that mood changes may occur on exposure to reminder of or discussion about the stressor referred to as “cognitive engagement” (Casey and Bailey, 2011). Although the DIAD questions on distress specifically ask to report symptoms present in the past seven days, it cannot be excluded that cognitive engagement with stressing events has biased responses.

Third, the PREDIS cohort study response rate was only 24.3%. This could have led to selection bias. We found no significant differences between responders and non‐responders as to gender and prevalence of certified ICD‐10 somatic and mental disorder. However, we found respondents to be significantly older than than non‐responders. In general, poor mental health is prevalent at all ages with the highest prevalence occurring in the youngest age groups (WHO International Consortium in Psychiatric Epidemiology, 2000). Prevalence rates of mental disorders found in the present study may therefore be an underestimation when compared with non‐responders. However, we found no significant difference between the PREDIS cohort and the target population, i.e. the national population of disability claimants as to the prevalence of somatic and mental disorders, certified by the SSI. Therefore, we believe our results as to the construct validity of the DIAD to be externally valid. Fourth, the capability of the DIAD to differentiate between persons with (n = 24) and without AD may be compromised by the small sample size.

Recommendations for future research

The present study describes the development of a new instrument and is a first effort to validate it. Further reliability and validity studies are clearly needed. Guidance for this validation process is provided by the consensus based standards for the selection of health status measurement instruments, i.e. the COSMIN checklist (Mokkink et al., 2010).

Content

The content of the DIAD should be further validated, with regard to the inclusion of AD subtypes depressed mood and anxiety, and recall bias. Allowing the DIAD to subtype AD is clinically relevant for treatment purposes. If the DIAD is aimed to be used as stand‐alone instrument, adequate subtyping can be achieved by including not only the 4DSQ subscale distress, but also the subscales depression and anxiety. If the DIAD is used in conjunction with a more comprehensive interview capable of detecting other mental disorders, e.g. the CIDI, then AD can be subtyped based on the diagnosis of subthreshold mood and anxiety disorders, diagnosed by the larger interview. For a clear differentiation between AD and depression the DIAD should be used in combination with a larger structured psychiatric interview, e.g. the CIDI, capable of detecting other DSM‐IV classifications.

To minimize recall bias, almost inherent to strict time limits in a diagnosis, in future versions of the DIAD, questions should be included about some other independent dateable events, e.g. related to sick leave, school attendance, employment, marriage, child birth, moving house, etc., that can be linked to self‐report dates of onset and termination of stressing events and symptoms.

Reliability and validity

Reliability of the DIAD should be assessed through test–retest and interrater reliability studies. Concurrent validity should be assessed by comparing DIAD outcome with those of clinical psychiatric interviews that include the diagnosis AD. The use of a clinical interview as gold standard is to be preferred above semi‐structured psychiatric interviews, such as the Schedules for Clinical Assessment in Neuropsychiatry (SCAN), the Structured Clinical Interview for DSM (SCID) or the Mini International Neuropsychiatric Interview (MINI), since the capability of these schedules to diagnose AD is limited (Casey, 2009). Longitudinal studies are needed to evaluate the predictive validity of the DIAD, i.e. to assess whether the DIAD correlates with some relevant criterion measure. To further investigate the capability of the DIAD to differentiate between persons with and without AD, future studies require clearly larger sample sizes. It is very important that psychometric properties, i.e. internal consistency, sensitivity, specificity, positive and negative predictive value, of the DIAD are assessed in other settings and populations, using appropriate recall periods, e.g. in community samples, primary care patients, psychiatric inpatients and outpatients, consultation liaison psychiatry and other groups of specific interest, such as those with deliberate self‐harm, sick listed or unemployed workers, high risk groups, or other specific age groups.

Conclusion

The expert consultation group was in moderate to good agreement on the content of AD, although whether the DIAD covers all essential aspects of AD is still not fully clear. Our hypothesis regarding the construct validity of the DIAD, was confirmed. These results are a first indication that the DIAD using adjusted DSM‐IV criteria is a valid, stand‐alone instrument to diagnose AD, to be administered by lay interviewers. With regard to the bereavement criterion, the DIAD is compatible to the upcoming DSM‐V. Further studies on criterion validity and reliability of the DIAD in other samples and settings are clearly needed. With a reliable and valid diagnostic instrument, the epidemiology of AD can be better researched and evidence‐based strategies for therapy and intervention can be developed.

Declaration of interest statement

The authors have no competing interests.

A the Diagnostic Interview Adjustment Disorder (DIAD)1

(When asking the questions the interviewer emphasizes the underlined words.)

Text to read out 1

In their lives, people may experience events or circumstances that cause stress. I want to ask you now if such problem situations or events exist or have occurred in the past. This refers to the previous three years, including the year prior to your calling in sick. Take your time to reflect.