Psychometric properties of the Mental Health and Social Inadaptation Assessment for Adolescents (MIA) in a population‐based sample

Abstract

We report on the psychometric properties of the Mental Health and Social Inadaptation Assessment for Adolescents (MIA), a self‐report instrument for quantifying the frequency of mental health and psychosocial adaptation problems using a dimensional approach and based on the DSM‐5. The instrument includes 113 questions, takes 20–25 minutes to answer, and covers the past 12 months. A population‐based cohort of adolescents (n = 1443, age = 15 years; 48% males) rated the frequency at which they experienced symptoms of Attention Deficit Hyperactivity Disorder (ADHD), Conduct Disorder, Oppositional Defiant Disorder, Depression, Generalized Anxiety, Social Phobia, Eating Disorders (i.e. DSM disorders), Self‐harm, Delinquency, Psychopathy as well as social adaptation problems (e.g. aggression). They also rated interference with functioning in four contexts (family, friends, school, daily life). Reliability analyses indicated good to excellent internal consistency for most scales (alpha = 0.70–0.97) except Psychopathy (alpha = 0.46). The hypothesized structure of the instrument showed acceptable fit according to confirmatory factor analysis (CFA) [Chi‐square (4155) = 9776.2, p = 0.000; Chi‐square/DF = 2.35; root mean square error of approximation (RMSEA) = 0.031; Comparative Fit Index (CFI) = 0.864], and good convergent and discriminant validity according to multitrait‐multimethods analysis. This initial study showed adequate internal validity and reliability of the MIA. Our findings open the way for further studies investigating other validity aspects, which are necessary before recommending the wide use of the MIA in research and clinical settings.

1. INTRODUCTION

The fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM‐5) introduces the dimensional approach to diagnosis and classification. While previous editions of DSM used a strictly categorical model to determine the presence or absence of a disorder, the dimensional approach has two main advantages for the assessment of psychopathology. First, it allows more flexibility in clinical contexts to assess the severity of a condition without implying a threshold between normality and pathology. Second, it allows more precision in research contexts for quantifying the disorders in terms of symptom count and for conducting analyses with continuous outcomes. This continuous quantification of mental health symptoms is particularly useful in community samples, where 12 months prevalence rates of disorders meeting full diagnostic criteria are low (Costello, Copeland, & Angold, 2011), and where subclinical levels of problems may be associated with lower levels of psychosocial functioning.

Dimensional self‐report questionnaires are available to researchers wanting to assess mental health among community samples of adolescents, such as the Youth Self‐Report (YSR; Achenbach & Rescorla, 2000) form or the Strength and Difficulties Questionnaire (SDQ; Goodman, 1997). The YSR (Achenbach & Rescorla, 2000) assesses eight syndromes (anxious/depressed, withdrawn/depressed, somatic complaints, social problems, thought problems, attention problems, rule‐breaking behaviour, and aggressive behaviour) and was shown to have acceptable fit indices (Ivanova et al., 2007) and Cronbach's alpha values (e.g. between 0.69 and 0.85 in an American sample; Ebesutani, Bernstein, Martinez, Chorpita, & Weisz, 2011). The SDQ (Goodman, 1997) has a five‐factor structure (emotional, conduct, hyperactivity‐inattention, peer relationship, and prosocial factors; see He, Burstein, Schmitz, & Merikangas, 2013) and an adequate mean Cronbach's alpha of 0.73. However, these instruments were not specifically designed to assess the 12‐months frequency of mental health symptoms that make up DSM‐5 diagnoses as well as common social adaptation problems of adolescence (e.g. low level delinquency, relational aggression). There is a need for DSM‐based dimensional assessment covering time spans often used on longitudinal studies (i.e. 12 months) and the entire spectrum of symptoms that make up common psychological symptoms of adolescence.

Multi‐informant assessment of child and adolescent psychopathology provides a more comprehensive view than that provided by a single rater (Martel, Markon, & Smith, 2017), but self‐rated measures are particularly important in adolescence. Indeed, as children become adolescents, self‐rated assessments of internalizing and externalizing symptoms become more accurate than ratings by a teacher or parent. In terms of internalizing symptoms, adolescents' capacity for introspection allows them to more reliably assess their own emotions (Berg‐Nielsen, Vika, & Dahl, 2003; Klaus, Mobilio, & King, 2009; Salbach‐Andrae, Klinkowski, Lenz, & Lehmkuhl, 2009; Swanson et al., 2014). In terms of externalizing problems, covert behaviours hidden from adults (e.g. stealing, vandalism) become more prevalent and can be reliably assessed via adolescent self‐reports (Augenstein et al., 2016).

1.1. The Mental Health and Social Inadaptation Assessment for Adolescents

We designed the Mental Health and Social Inadaptation Assessment for Adolescents (MIA) in response to limitations of existing tools. It provides a brief assessment of the symptoms that make up some DSM‐5 psychiatric disorders, and of related problems of social adaptation ranging from minor delinquency to severe physical violence, and relational aggression. An initial pool of items was created comprising symptoms assessed in common computer‐based tools such as the Diagnostic Interview Schedule for Children (DISC) (Shaffer et al., 1996) and the Development and Well‐being Assessment (DAWBA) (Goodman, Ford, Richards, Gatward, & Meltzer, 2000) as well as items used in dimensional instruments (the Behaviour Questionnaire, Tremblay, Desmarais‐Gervais, Gagnon, & Charlebois, 1987; the Child Behaviour Checklist, Achenbach & Rescorla, 2000; and the SDQ, Goodman, 1997). Item selection was based on both the adequacy of DSM‐5 criteria and the content validity (i.e. how well the items represent the concept under study) as assessed by the experts. All items were adapted to fit the 12‐month time frame and the self‐report format of the MIA. We chose the 12‐month time frame because it minimizes memory bias, allows the assessment of general tendencies to experience symptoms (rather than transient problems) and represents an interval often used in child and adolescent cohort studies.

The MIA includes four scales pertaining to internalizing disorders: Social Phobia, Generalized Anxiety, Depression, and Self‐harm, and six scales pertaining to externalizing disorders: Attention Deficit Hyperactivity Disorder (ADHD) (having three subscales: inattention, impulsivity, and hyperactivity), Conduct Disorder (having four subscales: lying, stealing, breaking rules, and vandalism), Psychopathy, Oppositional Defiant Disorder, Aggression (having four subscales: proactive aggression, reactive aggression, social aggression, and severe physical violence), and Delinquency and Contact with the Police. The last scale assesses Eating Disorder. Four scales assess interference with functioning for (a) Anxiety, (b) Depression, (c) Behaviour Problems, and (d) Eating Disorders in four contexts (i.e. family, school, peer relationships, and everyday life).

1.2. Aim of the study

The objective is to describe the psychometric properties of the MIA in a representative population‐based sample of adolescents using (i) Cronbach's alpha, to assess scales' internal consistency, (ii) t‐tests and effect size to describe sex differences, (iii) Spearman's rank correlation coefficient, to assess intercorrelations among scales, (iv) CFA, to test the internal structure of the instrument, and (v) correlation analysis inspired from the multitrait‐multimethods (MTMM) matrix, to assess internal convergent and discriminant validity of the scales.

2. METHODS

2.1. Participants

2.1.1. Pilot study

A pilot study was conducted in the fall of 2013 to obtain data on the acceptability of the MIA among a community sample of adolescents as well as preliminary psychometric properties prior to its use in a population‐based cohort. Participants in this pilot study were 311 adolescents (mean age 15.5 years; N = 126 males, 38%) recruited in three different neighbourhoods of a medium size North American city (Montreal, Canada) using different resources (posters, bookmarks, Facebook, newspaper advertisements). The pilot phase confirmed the good acceptability of the MIA and adequate functioning of the items: i.e. no items with extreme floor or ceiling effects, or unexpected number of missing data.

2.1.2. Main study

When they were 15 years old, participants in the Quebec Longitudinal Study of Child Development (QLSCD), a 15‐year representative population‐based longitudinal study, were invited to fill out the MIA. The instrument was administered in either French or English, as it was simultaneously developed in both languages. Participants in the QLSCD were selected via the Quebec Birth Registry using a stratified procedure based on living area and birth rate. QLSCD protocol was approved by the Quebec Institute of Statistics (Quebec City, Quebec, Canada) and the Sainte‐Justine Hospital Research Centre (Montreal) ethics committees. Written informed consent was obtained from all participants and parents at each data collection time. The initial sample included a total of 2120 infants representative of children born in the province of Quebec in 1997–1998 and followed prospectively until 15 years of age. The final sample consisted of the 1443 adolescents (mean age = 15.1 years; N = 691 males, 47.8%; see also Supporting Information Figure S1) participating in the 15 years collection time. Table 1 presents the sample characteristics.

Table 1.

Sociodemographic characteristics of the sample

	Sample (N = 1443)
Age, years [mean (SD)]	15.1 (0.3)
Male sex, N (%)	691 (47.8)
Country of origin, N (%)
Canadian	1066 (70.9)
Others	731 (32.9)
Maternal educational attainment, N (%)
High school diploma or less	583 (40.5)
Post‐secondary diploma	426 (29.5)
University diploma	434 (30.0)
Paternal educational attainment, N (%)
High school or less	698 (48.5)
Post‐secondary diploma	388 (26.8)
University diploma	357 (24.7)
Mode of living, N (%)
With both biological parents	827 (57.3)
With the mother	299 (21.0)
With the mother and her partner	211 (14.6)
With the father and his partner	33 (2.3)

2.2. Measure

The MIA includes 113 questions representing DSM‐5 symptoms for a given disorder (Supporting Information Appendix 1, English version, and Appendix 2, French version). Items of the psychopathology scales are answered on a three‐point Likert‐type scale (“never true”, “sometimes true”, “always true”), except for two out of three items of the Self‐harm scale, which are dichotomous. The score for each scale and subscale are obtained by computing the mean score of the corresponding items. The ADHD scale is calculated as the mean of impulsivity, inattention, and hyperactivity; the Conduct Disorder scale is calculated as the mean of lying, stealing, rule breaking, and vandalism; the Aggression scale is calculated as the mean of violence, proactive aggression, reactive aggression, and social aggression. A total internalizing score is obtained by computing the mean score of the Social Phobia, Generalized Anxiety, and Depression items. A total externalizing score is obtained by computing mean score of the ADHD, Conduct Disorder, Oppositional Defiant Disorder, Delinquency, and Aggression. A functioning impairment score is obtained by computing the mean of interference items (which are rated on a four‐point Likert scale: “not at all”, “slightly”, “somewhat”, “a lot”) in each of the family, school, friendship and daily functioning contexts. An interference with functioning score is also calculated for each type of problem: Anxiety, Depression, Behaviour Problems, and Eating Disorder. Finally, a total functional impairment score can be calculated as the mean of the four functioning impairment scales.

2.3. Statistical analysis

Statistical analysis were performed using R version 3.1 (R Core Team, 2014) and Mplus version 7.4 (Muthén & Muthén, 1998–2015).

2.3.1. Reliability

Internal consistency of each MIA scale was assessed by a version of Cronbach's alpha taking into account the ordinal nature of the items (Gadermann et al, 2012; Zumbo, Gadermann, & Zeisser, 2007). Values lower than 0.70 were considered “unsatisfactory”, between 0.70 and 0.79 “fair”, between 0.80 and 0.89 “good”, and ≥0.90 “excellent” (Cicchetti, 1994).

2.3.2. Sex differences and intercorrelations

We described the distribution of scores on each scale (e.g. Conduct Disorder) and subscale (e.g. Rule Breaking) using mean and standard deviation. Comparisons between the sexes were made using t‐test. Sex differences were estimated using effect size (Hedge's g); values <0.20 are interpreted as “small”, 0.21–0.50 “medium”, 0.51–0.80 “large”, and >0.80 “very large” (Cohen, 1988). Intercorrelations among the MIA scales were computed for each sex using Spearman's rank correlation coefficient, in order to take into account the non‐normal distribution of the MIA scales.

2.3.3. Internal structure of the instrument

CFA (with weighted least square mean and variable adjusted Mplus estimator, WLSMV, taking into account the ordinal nature of the items) was used to examine the a priori defined internal structure of the MIA. A third‐order CFA model was fitted: the first‐order factors were internalizing and externalizing dimensions; the second‐order factors were the 11 scales of the MIA; the third‐order factors were the subscales of the ADHD, Conduct Disorders, and Aggression scales. For these analyses we excluded the Self‐harm scale because of its extremely skewed distribution and its low prevalence. The structure of the functional impairment scales was tested separately using a second‐order CFA, where the four latent factors representing the functioning impairment scales loaded on a global functional impairment factor. The fit of the CFAs were evaluated using the CFI (acceptable fit if >0.95, poor fit if <0.90, otherwise marginal), and the RMSEA (acceptable fit if <0.06; Hu & Bentler, 1999). For model estimated using the WLSMV estimator, Yu and Muthén (2002) identify the RMSEA as the best performing indices. We also reported the Chi‐square test, although we could not rely on it as it is highly affected by sample size. Instead, the relative Chi‐square (ratio Chi‐square/DF) is suggested to be less influenced by the sample size; acceptable relative Chi‐square values range from less than two (Ullman, 2001) to less than five (Schumacker & Lomax, 2004).

2.3.4. Convergent and discriminant validity

We used a procedure inspired by the MTMM (Campbell & Fiske, 1959) and based on correlation analyses to jointly assess internal convergent and discriminant validity. MTMM allowed computing and comparing both item‐total score correlations and inter‐items correlations among the items of the scales. For item‐total score correlations we computed for each scale, the mean correlation between each item of the scale and the total score of each scale. For instance, for the Depression scale, we computed the mean correlation between each depression item and the total score of Depression scale. For inter‐items correlations, we computed the mean correlation among the items of a given dimension (e.g. mean correlation of the items of Depression among themselves), as well as the correlations within the items of that dimension and the items of other dimensions (e.g. correlation between Depression and Social Phobia items, between Depression and Aggression items, etc.). We expected, for each scale, the mean correlation among the items of the same scale (intracorrelation, or convergent correlation) to be greater than the mean correlation among the items of different scales (intercorrelations, or discriminant correlation). As for CFA, we excluded the Self‐harm scale from these analyses and we took into account the ordinal nature of the items in our MTMM using the WLSMV estimator.

3. RESULTS

3.1. Item analysis

Descriptive statistics of the MIA items are presented in Supporting Information Table S1. This analysis showed that missing data rate was low and homogeneous among all psychopathology items: it varied from three missing responses (0.2%, e.g. item “I was too fearful or nervous”, Generalized Anxiety scale) to 10 missing responses (0.7%, item “I felt sad and unhappy”, Depression scale). Evidence of a floor effect was found for several items, mainly from the Delinquency scale, the Proactive Aggression subscale, and Severe Physical Violence subscale, and it is attributable to the infrequency of these behaviours in the adolescent general population. Missing data on the functioning impairment items varied from four (0.3%, e.g. item “At home, with your family” of the Behavioural symptoms scale) to 17 (1.2%, item “In your daily occupations (i.e. not able to do things or go places)” of the Depression symptoms scale).

3.2. Reliability

Cronbach's alphas were “excellent” (≥ 0.90) for five scales (Social Phobia, Conduct Disorder, Depression, Delinquency and Contact with the Police, Aggression) and three subscales (Stealing, Proactive Aggression, and Severe Physical Violence). Alphas were “good” (0.80–0.89) for three scales (ADHD, Generalized Anxiety, and Oppositional Defiant Disorder) and six subscales (Impulsivity, Lying, Rule Breaking, Vandalism, Reactive Aggression, and Social Aggression), and “fair” for two (Eating Disorders and Self‐harm) scales and two subscales (Hyperactivity and Inattention). The Psychopathy scale, however, showed an unsatisfactory alpha (0.46), which did not improve after removing the item with the lowest correlation with the total score. Externalizing and internalizing dimensions had both “excellent” Cronbach's alpha (≥ 0.90). The Cronbach's alphas were “excellent” (≥ 0.90) for four functioning scales (Behaviour, Depression, Eating Disorders symptoms and total functioning) and “good” (0.88) for one scale (Anxiety symptoms; see Table 2).

Table 2.

Internal consistency, descriptive statistics, and sex differences of the MIA scales and subscales

	N items	Alpha	Total sample	Males	Females	Effect size
			N = 1443	N = 691	N = 753
			M (SD)	M (SD)	M (SD)
Psychopathology scales
Social Phobia	8	0.90	2.40 (2.18)	1.85 (1.91)	2.91 (2.28)	0.50***
ADHD	16	0.89	2.94 (1.67)	2.84 (1.69)	3.03 (1.65)	0.11*
Impulsivity	6	0.84	2.78 (1.96)	2.71 (2.03)	2.85 (1.89)	0.07
Hyperactivity	4	0.70	2.61 (2.13)	2.60 (2.14)	2.62 (2.13)	0.01
Inattention	6	0.77	3.31 (1.97)	3.13 (1.91)	3.48 (2.01)	0.18**
Generalized Anxiety	9	0.86	4.10 (2.17)	3.27 (1.97)	4.86 (2.08)	0.78***
Eating Disorders	5	0.70	1.60 (1.66)	1.13 (1.35)	2.04 (1.79)	0.57***
Conduct Disorder	16	0.95	0.65 (0.90)	0.63 (0.90)	0.66 (0.91)	0.02
Lying	5	0.84	0.99 (1.27)	0.97 (1.25)	1.01 (1.28)	0.04
Stealing	5	0.90	0.22 (0.73)	0.24 (0.80)	0.21 (0.66)	–0.05
Rule Breaking	4	0.85	0.92 (1.54)	0.81 (1.41)	1.01 (1.65)	0.13*
Vandalism	2	0.87	0.29 (1.02)	0.43 (1.18)	0.17 (0.83)	−0.26***
Depression	8	0.90	3.48 (2.25)	2.63 (1.95)	4.26 (2.22)	0.78***
Self‐harm	3	0.72	0.37 (1.19)	0.16 (0.71)	0.56 (1.47)	0.34***
Delinquency & Contact with Police	5	0.91	0.22 (0.73)	0.31 (0.86)	0.14 (0.57)	−0.23***
Oppositional Defiant Disorder	9	0.84	2.35 (1.39)	2.35 (1.36)	2.35 (1.41)	0.00
Psychopathy	4	0.46	3.35 (1.73)	3.73 (1.83)	3.00 (1.55)	−0.43***
Aggression	18	0.96	0.74 (0.88)	0.84 (0.95)	0.65 (0.79)	−0.22***
Severe Physical Violence	5	0.92	0.26 (0.86)	0.37 (1.05)	0.16 (0.61)	−0.26***
Proactive Aggression	4	0.91	0.18 (0.72)	0.22 (0.77)	0.13 (0.67)	−0.12*
Reactive Aggression	3	0.84	0.66 (1.42)	0.97 (1.66)	0.37 (1.09)	−0.44***
Social Aggression	6	0.86	1.57 (1.42)	1.59 (1.43)	1.56 (1.41)	−0.02
Total of Internalizing Symptoms	28	0.94	3.04 (1.74)	2.35 (1.49)	3.67 (1.71)	0.82***
Total of Externalizing Symptoms	56	0.97	1.44 (0.93)	1.45 (0.96)	1.44 (0.90)	−0.01
Functional impairment scales
Anxiety	4	0.88	1.85 (2.21)	1.34 (1.85)	2.32 (2.40)	0.45***
Behaviour	4	0.93	0.64 (1.44)	0.68 (1.40)	0.62 (1.47)	−0.04
Eating disorders	4	0.94	0.59 (1.47)	0.31 (0.94)	0.86 (1.78)	0.38***
Depression	4	0.90	1.92 (2.27)	1.23 (1.86)	2.54 (2.43)	0.60***
Total functional impairment	16	0.94	1.25 (1.46)	0.89 (1.15)	1.58 (1.62)	0.49***

Note: The table reports the descriptive statistics for all the MIA scales, subscales, and dimensions. All scores were rescaled to be expressed on a scale from 0 to 10. ADHD, Attention Deficit Hyperactivity Disorder; M, mean; SD, standard deviation. Effect size based on Hedge's g; p values are based on t‐tests.

^* p < 0.05.

^** p < 0.01.

^*** p < 0.001.

3.3. Sex differences and intercorrelations

Females scored higher than males on the internalizing scales: Social Phobia, Generalized Anxiety, Depression, and Self‐harm, as well as on the Eating Disorders scale (Table 2). The effect sizes were medium to large (0.34–0.78). Males scored higher than females on most externalizing behaviours, such as Proactive, Reactive, and total Aggression, Psychopathy, Violence, and Breaking Rules, with small to medium effect sizes (0.12–0.43). Significant sex differences (medium to large effect sizes; 0.38–0.60) were found in all scales of interference with functioning except Interference of Behavioural problems, with females experiencing more interference than males.

The correlation analysis reported in Table 3 showed that the correlations among externalizing scales were higher than those among the internalizing scales (and vice‐versa) for both sexes (see Figure S2 for a graphical overview of the correlations).

Table 3.

Intercorrelations between the MIA scales for boys (upper diagonal) and girls (lower diagonal)

Study sample	1	2	3	4	5	6	7	8	9	10	11	12	13	14
1.Social Phobia		0.37	0.51	0.25	0.24	0.50	0.14	0.07	0.21	−0.04	0.20	0.77	0.34	0.37
2.ADHD	0.32		0.50	0.35	0.52	0.65	0.22	0.30	0.57	0.06	0.56	0.61	0.88	0.53
3.Generalized anxiety	0.48	0.55		0.38	0.27	0.64	0.23	0.18	0.30	−0.15	0.30	0.87	0.45	0.57
4.Eating disorders	0.31	0.37	0.49		0.26	0.41	0.11	0.16	0.30	0.06	0.30	0.41	0.37	0.32
5.Conduct disorders	0.12	0.58	0.36	0.32		0.40	0.17	0.43	0.58	0.20	0.60	0.36	0.74	0.44
6.Depression	0.42	0.65	0.74	0.49	0.40		0.28	0.23	0.45	0.03	0.43	0.85	0.61	0.63
7.Self‐harm	0.19	0.26	0.33	0.21	0.27	0.38		0.10	0.14	0.02	0.13	0.28	0.20	0.31
8.Delinquency	0.05	0.23	0.13	0.10	0.33	0.12	0.15		0.32	0.12	0.38	0.20	0.45	0.28
9.Oppositional Defiant Disorder	0.21	0.62	0.39	0.31	0.53	0.48	0.28	0.24		0.26	0.70	0.38	0.80	0.44
10.Psychopathy	0.14	0.14	0.05	0.10	0.14	0.17	0.16	0.07	0.22		0.14	−0.06	0.17	−0.01
11.Aggression	0.18	0.56	0.33	0.30	0.54	0.4	0.21	0.26	0.61	0.03		0.37	0.76	0.42
12.Total internalizing	0.74	0.60	0.89	0.51	0.35	0.85	0.41	0.13	0.43	0.14	0.36		0.56	0.63
13.Total externalizing	0.27	0.90	0.51	0.41	0.76	0.61	0.30	0.32	0.79	0.17	0.68	0.55		0.56
14.Total functional impairment	0.36	0.58	0.69	0.48	0.49	0.72	0.42	0.24	0.52	0.18	0.41	0.71	0.62

Note: ADHD, Attention Deficit Hyperactivity Disorder.

3.4. Internal structure of the MIA

As the Psychopathy scale showed low reliability, it was excluded from the CFA. The model showed a good fit according to the RMSEA [0.31, 90% confidence interval (CI) = 0.030–0.031] although the CFI was below the suggested cutoff (0.864). As expected, the Chi‐square test was statistically significant (Chi‐square = 9776.21, DF = 4155, p < 0.001), although the Chi‐square/DF ratio (2.35) suggested the acceptability of the model. As reported in Table S2, all standardized factor loadings were superior to 0.3, except for the item “I completed all of my tasks or homework, I was able to stay focused” (subscale ADHD‐Inattention; 0.273). The CFA testing the structure of the functional impairment scales showed a good fit (Chi‐square = 468.23, DF = 99, p < 0.001; Chi‐square/DF = 4.73, CFI = 0.980; RMSEA =0.051, 90% CI = 0.046 − 0.056).

3.5. Convergent and discriminant validity

As shown in Table 4 and Figure 1, for most scales, items belonging to that scale had higher correlations with its total score compared to the total score of other scales, indicating good differentiation between scales. However, the items belonging to the Generalized Anxiety scale correlated similarly with the score of the Generalized Anxiety scale (0.56) and with that of the Depression scale (0.57). This was also true for items belonging to the Eating Disorders scale, which correlated similarly with the total score for Eating Disorders (0.37) and that of Depression (0.39). Concerning externalizing scales, items belonging to Oppositional Defiant Disorder correlated similarly with the total score of the other externalizing scales (0.37–0.49). Finally, items belonging to the Psychopathy scale presented low correlations (< 0.2) with the total score of all scales, including Psychopathy.

Table 4.

Item‐total scores correlations using multitrait‐multimethod correlations matrix of the psychopathology and functioning scales

	Part A. Psychopathology scales
	1.Social Phobia	2.Generalized Anxiety	3.Depression	4. Eating Disorders	5.ADHD	6.Conduct Disorder	7.Delinquency	8.ODD	9.Aggression	10.Psychopaty
1	0.62	0.38	0.39	0.24	0.20	0.13	0.11	0.13	0.13	0.02
2	0.44	0.56	0.57	0.39	0.31	0.20	0.11	0.19	0.15	−0.08
3	0.41	0.51	0.64	0.39	0.38	0.28	0.18	0.26	0.24	0.02
4	0.25	0.34	0.39	0.37	0.23	0.22	0.17	0.19	0.19	0.02
5	0.27	0.35	0.48	0.28	0.52	0.43	0.35	0.41	0.43	0.07
6	0.12	0.18	0.29	0.21	0.34	0.48	0.39	0.36	0.41	0.11
7	0.05	0.06	0.10	0.09	0.17	0.29	0.34	0.19	0.27	0.08
8	0.16	0.22	0.34	0.23	0.39	0.44	0.37	0.42	0.49	0.17
9	0.11	0.15	0.25	0.17	0.34	0.41	0.36	0.41	0.49	0.10
10	0.01	−0.09	0.03	0.02	0.05	0.15	0.18	0.16	0.16	0.14

Part B. Functional impairment scales
	1.Anxiety	2.Behaviour	3.Depression	4. Eating Disorders
1	0.64	0.33	0.58	0.33
2	0.41	0.58	0.41	0.26
3	0.60	0.34	0.68	0.35
4	0.44	0.27	0.43	0.58

Note: The correlation matrix in Part A represents the correlations between the items of each scale (row) with the total score of each scale (columns). Items of a given scale should have higher correlation with the total score of their scale (i.e. values in the diagonal of the matrix; in bold typeface) than with the other scales (values outside the diagonal). The correlation matrix in Part B represents the mean correlation of the items of a given scale with the items of each scale. Items belonging to the same scale should correlate highly among themselves (i.e. values in the diagonal of the matrix; in bold typeface). Items belonging to different scales should not correlate highly (values outside the diagonal).

ADHD, Attention Deficit Hyperactivity Disorder; ODD, oppositional defiant disorder.

Figure 1.

Item‐total scores correlations from the multitrait‐multimethod analysis.

Each boxplot indicates the correlation (reported in the y‐axis) between the items of a given scale (reported in the x‐axis) and the total score of a given scale (reported in the title of each panel). The grey box indicates the correlation between the total score indicated in the title of the panel and the items of that scale. For each panel, the grey box is expected to show the highest correlation (i.e. graphically, to be over the others boxes).

For instance, the first panel reports the correlations between the items of 10 MIA scales and the total score of Social Phobia. The correlation between the items composing the Social Phobia scale correlated higher with Social Phobia total score (grey box) than the items from the other scales (white boxes). ADHD=Attention Deficit Hyperactivity Disorder

Overall, the correlation between items pertaining to the internalizing scales correlated more strongly with the total scores of these scales than with the total scores of the externalizing scales. The Eating Disorder scale (whose items correlated with the Eating Disorder total scores at 0.37) was less differentiated from the other scales, especially Depression (0.39) and Generalized Anxiety (0.39).

Concerning the four functioning scales, the correlations between items belonging to each functioning scale and their total score ranged from 0.58 (Interference of Behaviour and Eating Disorder with functioning) to 0.68 (Interference of Depression with functioning), and were lower with the total score of different scales. The magnitudes of the correlations were similar between the items from the Anxiety and Depression interferences scales.

Inter‐items correlation analysis showed results consistent with the item‐total score correlation analysis (Table S3 and Figure S3).

4. DISCUSSION

The aim of the present study was to describe the psychometric properties of the MIA, a self‐report questionnaire assessing DSM‐5 symptomatology and social adaptation problems using a dimensional approach. The dimensional approach is in line with the addition of a dimensional diagnosis to the fifth edition of the DSM. It confers advantages over the categorical approach for examining the extent to which low or moderate number of symptoms are associated with poor outcomes in a dose‐response fashion; as well as for testing potential gradients in the associations between symptoms and biomarkers (Kraemer, 2015).

A distinctive feature of the MIA pertains to the breath of the problems it covers, including both internalizing and externalizing symptoms as well as a comprehensive range of antisocial and aggressive behaviours. The instrument allows assessing both normative behaviours often associated with mental health problems (e.g. social aggression, impulsivity) and common deviant behaviours of adolescence (e.g. truancy from school, occasional stealing). As such, the MIA may be a useful instrument for researchers and clinicians wanting to obtain an overview of adolescent functioning over the past 12 months in a brief time span (20–25 minutes).

Our results indicate good to excellent internal consistency (Cronbach's alpha) for most MIA scales and subscales, including those having few items (e.g. Impulsivity). As Cronbach's alpha tends to be lower with lower number of items, these findings corroborate the reliability of the MIA scales. Alpha values were above 0.9 for some scales (Delinquency, Conduct Disorder, Aggression, Total Internalizing, and Total Externalizing scales) and subscales (Severe Physical Violence and Proactive Aggression) which may indicate redundancy and the need for removing some items (Tavakol & Dennick, 2011). However, we note that high alphas on some of these scales can be explained by the high number of items that they comprise of (Conduct Disorder: n = 16; Aggression, n = 18; Total Internalizing: n = 28; Total Externalizing; n = 56). For the Delinquency scale (as well as for Severe Physical Violence and Proactive Aggression subscales), the high Cronbach's alpha despite the low number of items may be due to the dichotomous distribution of responses in the population: i.e. some individuals presenting with a wide range of delinquent behaviours, and others showing no such behaviours.

Unlike the other scales, the Psychopathy scale showed a low Cronbach's alpha (0.46). The multidimensionality and heterogeneity of antisocial behaviours in non‐clinical samples, and the low prevalence of psychopathic traits in the general population may explain this finding (Colins, Fanti, Salekin, & Andershed, 2016; Salekin, 2016; Werner, Few, & Bucholz, 2015). Future studies investigating the psychometric properties of the Psychopathy scale among individuals exhibiting antisocial behaviours are needed.

The sex differences on the MIA scales were in the expected direction: females scored higher than males on internalizing scales (e.g. Depression and Generalized Anxiety), while males scored higher on externalizing scales (e.g. Conduct Disorder and Aggression). Females experienced more interference than males on depression, anxiety, and eating disorder, while males experienced more interference on the behaviour problem scale. The effect sizes for the sex differences were medium to large (i.e. 0.38–0.60) and are consistent with the sexual dimorphism of internalizing and externalizing disorders reported in previous studies (Côté, Vaillancourt, Barker, Nagin, & Tremblay, 2007; McLean & Anderson, 2009; Parker & Brotchie, 2010; Rescorla et al., 2012).

CFA provided evidence for the validity of both higher‐ and lower‐order structure of our model (i.e. internalizing and externalizing dimensions, as well as specific psychopathology scales and subscales). However, the CFI index was below the suggested cutoff, indicating the need for further improvement. Of note, relying on the CFA is often considered an over‐restrictive approach for the evaluation of questionnaires with complex structures (i.e. multiple scales with multiple subscales) because it constraints each item to correlate only with one factor (i.e. imposing zero cross‐loading). This is a particularly restrictive assumption when applied to complex phenotypes such as psychopathology (or personality, Booth & Hughes, 2014; Marsh, Morin, Parker, & Kaur, 2014; Marsh, Nagengast, & Morin, 2013), where the co‐occurrence of symptoms is the norm rather than the exception, and where multiple scales may each assess a distinct facet of a given phenomenon. In addition, the MIA internalizing and externalizing symptomatology is drawn from clinically meaningful entities (proposed in the DSM) and as such provides an assessment of a priori defined concepts. Thus, the concepts assessed were not empirically derived and may not cluster together as would be expected if a data‐driven approach was used to identify clusters of symptoms (Hartman et al., 2001; Wakefield, 1999). We also note that lower CFI values were reported in previous studies using CFA on multidimensional instruments (Booth & Hughes, 2014) such as the Youth Self‐Report (Achenbach & Rescorla, 2000; Ivanova et al., 2007). In such multidimensional instruments, RMSEA index of fit may be considered more reliable in analysis based on the WLSMV estimator (Yu & Muthén, 2002), and was used as primary fit index, together with CFI values >0.80 (Ivanova et al., 2007).

In this study we corroborated the CFA findings with a MTMM analysis. The MTMM approach allows each item to correlate with each psychopathology factor (i.e. each total score), thereby providing a more sensible representation of the correlations between items and dimensions in a context of comorbidity. The MTMM analysis showed satisfactory internal convergent and discriminant validity: i.e. item total and inter‐items correlations were higher for the items belonging to the same scale – corroborating convergent validity, than for the items belonging to different scales – corroborating discriminant validity. However, the Eating Disorders scale was poorly distinguished and correlated moderately with both the Depression and Generalized Anxiety items and total scores. This can be related to the particularly high level of co‐occurrence between eating, depressive, and anxious symptomatology (Godart et al, 2003; Kaye, Bulik, Thornton, Barbarich, & Masters, 2004; Treasure, Claudino, & Zucker, 2010). Similarly, Psychopathy items had low correlations with both the Psychopathy total score and the other scales' total scores. However, MTMM analysis showed that, although poorly correlated among themselves, the Psychopathy items were more correlated with Delinquency, Aggression, and Conduct Disorder, than with internalizing symptomatology (in particular Generalized Anxiety and Depression). This finding supports the discriminant validity of the Psychopathy scale with respect to the internalizing psychopathology.

MTMM analyses also showed that the items belonging to the externalizing psychopathology scales (i.e. ADHD, Conduct Disorders, Delinquency, Oppositional Defiant Disorder, and Aggression) were more correlated with the total scores of these subscales than with the total scores of the internalizing scales (i.e. Social Phobia, Generalized Anxiety, Depression). This expected pattern supports the differentiation between the internalizing and externalizing dimensions (Achenbach, Ivanova, Rescorla, Turner, & Althoff, 2016) tested in the CFA, a distinction which has also been recently endorsed by the DSM‐5 (Regier, Kuhl, & Kupfer, 2013).

4.1. Strengths and limitations

The first strength of this study is the use of a large and representative sample of adolescents (n = 1443) for the description of the psychometric properties of a questionnaire providing a broad and brief assessment of mental health and social adaptation. The second is the use of two distinct methodological approaches to investigate the internal validity of the instrument: a more restrictive and classical approach – CFA – as well as a less restrictive approach – the MTMM matrix. Both approaches supported the adequate psychometric properties of the scale. Thirdly, our analyses accounted for the ordinal nature of the items, which is often an overlooked aspect in validation studies and may lead to deflated reliability estimates (Gadermann et al, 2012; Zumbo et al., 2007).

Note of caution and limitations should be made. First, like any other self‐report scale, the MIA cannot replace assessments performed by a clinician. It is an instrument designed for use with community or clinical samples of adolescents in a research context, or to complement clinical assessments in a clinical context. The MIA relies on self‐reports, as this is the method of choice for collecting mental health information in adolescent community samples. There is solid evidence that youths with and without significant mental health problems understand and have insight on their difficulties and that they can provide unique and valid information on their mental states (Martel et al., 2017). This is especially the case for conduct problems of illegal nature (e.g. stealing), which are most often hidden from adults and reliably assessed via self‐reports (Deighton et al., 2014). In addition, self‐reports can be performed with low costs of administration and do not involve relying on reporters typically solicited in population‐based studies (such as teachers) and for which low responses rates are often obtained. Second, our analysis concerns the internal validation of the MIA. Convergent and discriminant validity of the MIA against external criteria, as well as the test–retest reliability should be tested in future studies. Third, although it is important to assess the MIA properties in general population samples, examination of the psychometric properties among clinical populations would provide useful information for future use with such populations. Finally, the MIA does not assess all form of adolescent psychopathology but rather focuses on some of the most prevalent mental health and social adaptation problems among community samples. These limitations open the way for future studies, especially those examining a more comprehensive range of psychometric properties.

DECLARATION OF INTEREST STATEMENT

The authors have no conflicts of interest to declare.

Supporting information

Supporting info item

Supporting info item

Table S1. Confirmatory Factor Analysis of the MIA: factor loadings

Table S2. Inter‐items correlations using multitrait‐multimethod matrix of the psychopathology and functioning scales

Côté SM, Orri M, Boivin M, et al. Psychometric properties of the Mental Health and Social Inadaptation Assessment for Adolescents (MIA) in a population‐based sample. Int J Methods Psychiatr Res. 2017;26:e1566 10.1002/mpr.1566