Achieving Depression Literacy: The Adolescent Depression Knowledge Questionnaire (ADKQ)

Abstract

Mental health literacy appears to be an important target for prevention and intervention efforts. However, limitations exist in this literature base, including the lack of a validated measure to assess this construct. The Adolescent Depression Knowledge Questionnaire (ADKQ) was created to assess knowledge of depression and attitudes about seeking help (i.e., depression literacy) for mental health issues before and after introduction of a universal, school-based intervention, the Adolescent Depression Awareness Program (ADAP). The ADKQ measured depression knowledge and attitudes in 8,216 high school students immediately before ADAP was implemented and 6 weeks after. The latent structure of the Knowledge section was examined with attention to measurement invariance between males and females and type of instructor, as well as pre- to post-test. Categories were developed for the open-ended questions of the Attitudes section. A one-factor (General Knowledge) latent structure was the best fit to the data. The latent structure of the ADKQ did not differ by student’s gender or type of instructor, nor did it differ based on pre- or post-test. Categories for the Attitudes portion of the ADKQ were developed. Psychometric evidence supports the ADKQ as a measure to evaluate adolescent depression literacy pre- to post-test and within several groups of interest (e.g., gender, facilitator). Categories for the Attitudes section of the ADKQ will allow for easier evaluation of this measure with quantitative data.

Introduction

Adolescent-onset mood disorders are frequently unrecognized or misdiagnosed and often go untreated (Christiana et al., 2000). Depression is estimated to affect approximately 8.5 % of adolescents in the United States (SAMHSA, 2008), with 20–50 % of all adolescents reporting clinically significant levels of depressive symptoms (Kessler, Avenevoli, & Merikangas, 2001). While less prevalent, it is estimated that bipolar disorder affects approximately 1.8 % of individuals younger than 18 years old (Van Meter, Moreira, & Youngstrom, 2011). In general, there appears to be an inverse relationship between age at diagnosis and treatment initiation for these mood disorders. That is, individuals who develop mood disorders during adulthood are nearly 14 times more likely to receive treatment within the year of onset as compared to those who develop mood disorders in childhood (Olfson, Kessler, Berglund, & Lin, 1998).

Untreated mood disorders may result in a variety of negative consequences, the most serious of which is suicide. Suicide ranks as the third leading cause of death in 10- to 24-year-olds (CDC, 2012) and was the leading cause of injury mortality in 2009 for all age groups combined (Rockett et al., 2012). A significant majority of suicides and suicide attempts (90 %) occur in the context of mental illness (Cavanagh, Carson, Sharpe, & Lawrie, 2003). Both unipolar depression and bipolar disorder are highly associated with suicidal thoughts and behaviors. In fact, these disorders represent some of the highest odds ratios related to suicide attempt and ideation among psychiatric diagnoses in adults (Nock, Hwang, Sampson, & Kessler, 2010). Early-onset mood disorders have been found to be strongly associated with suicidal thoughts and behaviors (Nock et al., 2010; Lewinsohn, Seeley, & Klein, 2003; Jonsson et al. 2011). Children and adolescents with bipolar disorder are at significantly greater risk than are those with major depression. These individuals tend to be younger at first attempt, make more lethal attempts, and are more likely to make multiple attempts (Goldstein et al., 2005; Lewinsohn et al., 2003).

Additionally, depressive symptoms in adolescence are associated with multiple deleterious outcomes in adolescence through adulthood. For example, developmental trajectories of youth with depressive symptoms indicate more frequent high-risk behaviors (e.g., multiple sexual partners, criminal history, excessive substance use), poorer coping with stress, greater loneliness, lower education completion and high school dropout, and continued mental health problems in adulthood (Jonsson et al., 2011; Lee et al., 2009; Quiroga, Janosz, Bisset, & Morin, 2013; Wickrama & Wickrama, 2010; Yaroslavsky, Pettit, Lewinsohn, Seeley, & Roberts, 2013). Adolescents with higher levels of depressive symptoms tend to have lower Grade Point Averages (GPA), lower standardized academic test scores, have higher rates of absenteeism, and express lower levels of desire to attend college (Jones, 2009). The relationship of depression with academic outcomes is a complex one, in that depression may be both a catalyst and a consequence for these academic challenges (Verboom, Sijtsema, Verhulst, Penninx, & Ormel, 2014). Regardless, depressive symptoms remain an important target for prevention and intervention. Likewise, children and adolescents with bipolar disorder are more likely to repeat a grade, require special education services, and are less likely to graduate on-time or complete higher education degrees than their peers (Biederman et al., 2005; Glahn, Bearden, Bowden, & Soares, 2006; Lagace & Kutcher, 2005; Wozniak et al., 2011).

Unfortunately, the vast majority of youth with mental health disorders and symptoms (approximately 75–80 %) do not receive mental health treatment (US Public Health Service, 2000). Early identification and treatment of mood disorders are essential. Studies show that treatment leads to decreased morbidity and mortality, including the reduction in suicidal thoughts and behaviors (e.g., Nery-Fernandes et al., 2012; Treatment of Adolescents with Depression Study [TADS] 2009). As such, there is a clear need for interventions which increase the recognition of early symptoms and the treatment of mood disorders among youth.

Mental Health Literacy in Adolescents

In the development of interventions, theories of behavior change must be considered. The Unified Theory of Behavior (UTB) emerged from a National Institute of Mental Health (NIMH) meeting intending to summarize and consolidate social and developmental psychology theories (Fishbein et al., 2001). Therefore, the UTB provides a comprehensive framework to understand behavior change. The UTB proposes that there are several precursors to behavior change, including willingness or intention to change and knowledge, skills, and abilities to use mental health services (Jaccard, Dodge, & Dittus, 2002). As such, research has generally demonstrated that negative attitudes and lack of knowledge about mental illness and mental health treatment have been identified as crucial barriers to seeking help (Bruffaerts et al., 2011). Additionally, educating individuals about mood disorders has been indicated as a way to enhance treatment utilization (Alonzo et al., 2011; Cusimano & Sameem, 2011; Fröjd, Marttunen, Pelkonen, von der Pahlen, & Kaltiala-Heino, 2007; Olsson & Kennedy, 2010).

Within this UTB framework, the concept of mental health literacy is illuminated. Jorm et al. (1997) first coined the term “mental health literacy” which includes the ability to recognize: a) signs and symptoms, b) risk factors and causes, c) attitudes that create barriers to or facilitation of help-seeking, and d) how to seek treatment for psychological distress or mental health disorders. Mental health literacy appears to be an important target for intervention. For example, it has been shown that overall, adolescents’ knowledge of mental health disorders is limited (Hess et al., 2004; Olsson & Kennedy, 2010). An adolescent’s knowledge of mental disorders is directly tied to help-seeking intentions (i.e., those who demonstrated more accurate knowledge of mental health disorders were more likely to report they would seek help for an individual demonstrating psychological distress (Olsson & Kennedy, 2010). Relatedly, it has been shown that improving mental health literacy tends to decrease levels of mental health stigma and increases self-reported help-seeking intentions. Importantly, those with more negative attitudes and feelings about mental illness are most affected by these increases in knowledge (Watson et al., 2004; Yap, Wright, & Jorm, 2011).

Most research regarding help-seeking and depression literacy has focused on self-reported help-seeking intentions rather than help-seeking behaviors. However, there has been prospective research indicating that as adolescents’ perceived need for help increases, they are more likely to seek help for depression (Fröjd et al., 2007). Interestingly, this research highlights the importance of significant others (i.e., parents and peers), as concerns about these individuals were the best predictors of help-seeking. Given the tendency of adolescents and young adults to seek advice and guidance from peers (e.g., Fröjd et al., 2007; Quinn, Wilson, MacIntyre, & Tinklin, 2009), the importance of all adolescents and young adults having accurate health information is emphasized.

There appear to be important gender differences in both knowledge and self-reported help-seeking intentions. For example, in a sample of adolescents and young adults, Cotton, Wright, Harris, Jorm, & McGorry (2006) found that females were significantly more likely to correctly identify depressive symptoms in a vignette than were males, and males were significantly more likely to endorse using alcohol as a way to deal with depression. Olsson and Kennedy (2010) demonstrated that female adolescents were more likely to indicate they would talk to a depressed peer and recommend getting help than were their male counterparts.

Taken together, this research emphasizes the long-term importance of improving mental health literacy. Schools have been identified as the ideal place to address mental health needs (Department of Health and Human Services, 1999; New Freedom Commission on Mental Health, 2003). Providing evidence-based programs within the school context increases the opportunity to reach adolescents, especially those who rarely see their primary care physicians or seek mental health treatment. When interventions are integrated into the typical school curriculum, students and their parents view them as another aspect of school, which can increase access to services and reduce stigma (Rivet-Duval, Hariot, & Hunt, 2011). However, a recent systematic review of the effectiveness of school-based mental health literacy programs identifies deficits in the current state of the literature (Wei, Hayden, Kutcher, Zygmunt, & McGrath, 2013). In their review of 27 articles, Wei and colleagues noted concerns with the majority of studies in this relatively new school mental health literacy field. Common limitations included lack of randomization, confounding factors, statistical deficiencies (e.g., missing data analysis), and lack of validated measures. While there appears to be promise in this field, these researchers have highlighted the need for further study.

ADKQ and the Current Study

To the best of our knowledge, there are no validated measures to assess the construct of depression literacy. In general, measures utilized in research on mental health literacy have been developed for a specific line of research and are not readily available for mental health professionals (e.g., Fröjd et al., 2007; O’Kearney, Kang, Christensen, & Griffiths, 2009; Olsson & Kennedy, 2010; Watson et al., 2004; Yap et al., 2011; Yap & Jorm, 2012). Herein, our goal is to analyze the psychometric properties of the Adolescent Depression Knowledge Questionnaire [ADKQ]. The ADKQ was designed in conjunction with the Adolescent Depression Awareness Program (ADAP), a school-based, universal program designed to educate high school students about mood disorders (i.e., increase “depression literacy” while reducing mental health stigma). The ADKQ consists of two sections (i.e., Knowledge and Attitudes) to evaluate student’s depression literacy. The ADKQ was developed to evaluate the effectiveness of ADAP (see Hess et al., 2004 and Swartz et al., 2010 for further discussion of ADAP). Aside from measuring pre- and post-test impact of the ADAP intervention, the ADKQ has broader utility to identify gaps in depression literacy that might be helpful in directing targeted interventions.

Our aims are twofold: Aim 1: examine the latent structure of the Knowledge section of the ADKQ and estimate the reliability of this section and Aim 2: code and analyze data from the Attitudes section of the ADKQ. We hypothesized that two or three factors will emerge in the Knowledge section. As the exploration of the Attitudes section was preliminary, no a priori hypotheses were predicted.

Method

Scale Development

The ADKQ was developed to measure changes in depression knowledge and attitudes, the primary targets of the ADAP intervention. These core components were determined to be critical elements essential to depression literacy. The items were developed through collaboration with many experts/researchers in the arena of mood disorders by focusing on the extant literature and their extensive background experiences and knowledge. The items were carefully selected from the literature to tap adolescent depression knowledge and attitudes. The core ADAP team, consisting of psychiatrists, psychiatric nurses, and medical students, evaluated the items. The scale developed over time through feedback from facilitators of, students receiving, and evaluators of ADAP.

The final ADKQ consists of 19 questions—15 comprising the Knowledge section (13 dichotomous and 2 fill in the blank, see Table 2) followed by 4 comprising the Attitudes portion (1 dichotomous and 3 open-ended; see Table 3). In previous studies (Hess et al., 2004; Swartz et al., 2010), depression literacy (i.e., a score of 80 % correct) has been investigated using the Knowledge portion of the ADKQ. One point is awarded for each correct response on the dichotomous items (i.e., 13 points) with seven points possible for listing symptoms of depression and mania (i.e., fill-in-the-blank items 14 and 15). While items 14 and 15 have been previously included in assessing pre- to post-test change, they are methodologically distinct from the other items and were excluded from analyses investigating the latent structure.

Table 2.

Knowledge section of the ADKQ and CFA factor loadings from the one-factor model (KNOW), items 1–13

	Item	Subsample 1 λ	Subsample 2 λ	Subsample 3 λ
1	Five percent of all teenagers will suffer a Major Depression	0.88	0.88	0.55
2	Major Depression is a normal part of adolescence	0.82	0.85	0.69
3	Depression runs in some families	0.91	0.92	0.64
4	Depression can be controlled through willpower	0.65	0.68	0.44
5	The cause of Major Depression is well known	0.71	0.73	0.63
6	A change in behavior is a symptom of depression	0.88	0.88	0.63
7	There are certain groups of people who are immune to depression	0.83	0.84	0.68
8	Major Depression is a treatable medical illness	0.80	0.81	0.48
9	A person with depression always feels sad	0.81	0.81	0.51
10	The abuse of alcohol and drugs can be a sign of depression	0.98	0.99	0.89
11	Bipolar Disorder is more common than Major Depression	0.63	0.63	0.40
12	Major Depression is a curable illness	0.52	0.54	0.22
13	Someone who has a major stress (like having parents get a divorce) always develops a depressive illness	0.84	0.85	0.75
14	List five symptoms of depressive illness	n/a	n/a	n/a
15	List two symptoms of mania	n/a	n/a	n/a

Subsamples 1 and 2 include post-test data, while subsample 3 includes pre-test data. Items 14 and 15 were not included in factor analyses

Table 3.

Items and categories developed for the Attitudes section of the ADKQ (with sample responses used to categorize)

Category and sample responses	n (%) Ever endorsed
16. If you thought that you or a friend was depressed would you encourage him/her to ask for help?
17. Who would you ask for help?
Family: “Parent/Guardian”; Family Member”	5,065 (59.4 %)
Mental Health/Medical Professional: “Counselor”; “Psychiatrist”, Mental Health Provider, Doctor, Other Health Provider”	4,072 (47.8 %)
School-Based Adults: “Teacher”; “School Nurse”; “Other School Personnel”	2,182 (25.6 %)
Peers: “Friend”; “Peers/Classmates”	1,344 (15.8 %)
Miscellaneous
”Adult”	2,346 (28.6 %)
”Someone you can trust/who can help/who listens”	479 (5.6 %)
”Clergy”	80 (0.9 %)
”Hotline”	25 (0.3 %)
”Other”	181 (2.1 %)
18. What would stop you or make it difficult for you to get help for yourself or a friend?
Emotional/embarrassment/stigma
“You might be embarrassed, thinking it doesn’t happen to everyone”; “Awkward topic to talk about”; “Scared of what people think”; “Embarrassment that I’d have to admit it; embarrassed for others to know”; “Knowing that people would make fun of me/them”	3,724 (43.7 %)
Social/interpersonal
“Fear of breaking my friend’s trust”; “My friend getting mad”; “Peer pressure”; “Wouldn’t want to get into their business if they didn’t want me to”; “I don’t want to break a promise to my friend”	3,341 (39.2 %)
Depression/treatment issues
“Treatment may not work”; “I have no one to go to for help”; “No one can help…don’t want help”; “Side-effects of medicine”	3,923 (46.0 %)
Nothing should get in the way	754 (8.8 %)
19. If your friend tells you that he/she is depressed and asks you to keep it a secret because no one else knows, what would you do?
Tell someone	5,477 (64.2 %)
Keep it a secret	3,146 (36.9 %)

Item 16 is dichotomous and therefore was not included in the coding analyses

Subjects

The study protocol was reviewed and approved by the Johns Hopkins School of Medicine Institutional Review Board (IRB). Participants included ninth grade students from thirty high schools across the United States enrolled in health education class between the fall of 2005 and spring of 2010 (N = 8,525). Unmatched pre- or post-tests (n = 309) were excluded, resulting in a sample of n = 8,216. As the study was human subjects exempt, very little identifying information was collected. Self-report of sex indicated that, of respondents who voluntarily offered this information (n = 8,070), 48.8 % were male. Within the sample of schools included in this study, there was geographic and socioeconomic diversity (Table 1).

Table 1.

Demographic characteristics of participant schools

School	N	% of Total sample	State	Free or reduced lunch (%)	School type
1	375	4.6	Maryland	6.9	Public
2	1,378	16.9	Maryland	30.3	Public
3	1,514	18.6	Maryland	4.0	Public
4	42	0.5	Maryland	–	Private secular
5	134	1.6	Washington, DC	–	Private secular
6	149	1.8	Maryland	–	Private Religious
7	194	2.4	Maryland	35.4	Public
8	63	0.8	Maryland	74.4	Public
9	1,023	12.5	Maryland	2.1	Public
10	334	4.1	Maryland	–	Private secular
11	271	3.3	Oklahoma	52.7	Public
12	117	1.4	Oklahoma	49.0	Alternative
13	163	2.0	Oklahoma	47.6	Public
14	158	1.9	Oklahoma	–	Private Religious
15	111	1.4	Oklahoma	99.7	Public/magnet
16	71	0.9	North Carolina	–	Private Religious
17	78	1.0	Maryland	25.2	Public
18	284	3.5	Oklahoma	85.9	Public/magnet
19	135	1.7	Oklahoma	82.0	Public/magnet
20	45	0.6	Maryland	–	Private religious
21	41	0.5	Maryland	–	Private religious
22	31	0.4	Oklahoma	80.0	Alternative
23	248	3.0	Maryland	–	Private religious
24	48	0.6	Delaware	–	Private religious
25	32	0.4	Delaware	28.0	Public
26	56	0.7	Delaware	41	Public
27	460	5.6	Texas	0.0	Public
28	348	4.3	Ohio	26.2	Public
29	58	0.7	Wisconsin	22.6	Public
30	200	2.5	Ohio	6.7	Public

Procedure

A variety of instructors were responsible for administration of ADAP with the given sample. Initially, guest facilitators (Johns Hopkins psychiatrists, psychiatric nurses, and nursing and medical students) served as instructors, teaching the curriculum in health classes in partnering schools. This group facilitated ADAP administered to n = 5,889 participants. Next, school-based educators—counselors and teachers—were trained to implement the program on their campus during health class. School professionals administered ADAP to n = 2,052 participants. All trainings were conducted by the same core ADAP team members and covered identical content. However, medical and nursing students received additional training and practice in teaching and classroom management, a module not included for school-based educators, who presumably already possessed those skills.

Students were given the ADKQ as a pre-test after a brief introduction to ADAP but prior to the intervention. Unique identification numbers were used by school personnel to match pre- and post-tests. The post-test was administered unannounced 6 weeks after the last session.

Data Analysis

The critical issues related to validating a measure are validity and reliability. Messick (1995a, b) argued that at the core of validity is construct validity (i.e., the support for the trustworthiness of results). Generalizability and predictive ability of the results are also essential elements of validity (Messick, 1995b). Items should be correlated with other, similar items, and the measure should predict other important behaviors. Determining the internal, or latent, structure of the measure is a critical component to this aspect of validity. This will be evaluated through factor analysis, a method that investigates relationships among items. Factor analysis provides for a more parsimonious understanding of a construct, as there are fewer factors than indicators.

A second issue regarding psychometric properties is that of reliability. Reliability refers to the extent that a scale provides consistent results. While there are several methods to assess reliability, internal consistency is the most widely used method (DeVellis, 2003; Furr & Bacarach, 2008). Internal consistency reliability is concerned with the homogeneity of the items within a scale (DeVellis, 2003). The idea behind this method is related to the dimensionality of a test (i.e., that the items of a test either measure similar—unidimensional, or different aspects—multidimensional; Furr and Bacarach 2008). That is, internal consistency reliability evaluates the variability within a set of items (DeVellis, 2003). After the latent structure is determined, internal consistency reliability will be investigated.

Aim 1: Investigating the Latent Structure of the ADKQ Knowledge Section

To examine the latent structure, confirmatory factor analysis (CFA) and measurement invariance analyses were conducted. Similar analyses were conducted by (1) sex (male/female), (2) type of facilitator (school-based, outside facilitator), and (3) time (pre versus posttest) for the analyses of measurement invariance. Therefore, several subsets of data were created on which CFA analyses were conducted followed by measurement invariance with these groups. These analyses were chosen in order to investigate the latent structure of this portion of the ADKQ (i.e., through CFA) and whether the structure is similar across subgroups of interest (i.e., through measurement invariance). These subgroups included (1) gender, (2) facilitator, and (3) time. As the data are categorical (i.e., items are scored as either correct = 1 or incorrect = 0), a robust weighted least squares estimator (WLSMV) was used. Mplus, version 7.0 (Muthén & Muthén, 1998–2012) was used for all analyses.

It was hypothesized that two or three ADKQ Knowledge factors would emerge (see Table 2). A two factor model included: Causes/Etiology—items 3, 4, 5, 7, 8, 11, 12, 13, and Signs/Symptoms—1, 2, 6, 9, 10. A three factor model included: Causes/Etiology—3, 4, 5, 8, 13, Signs/Symptoms—6, 9, 10, and Basic Knowledge—1, 2, 7, 11, 12. CFA was utilized to compare the fit of each of these models (i.e., two or three factor) with the null model (i.e., a one-factor model). Exploratory factor analysis (EFA) was not chosen as a priori hypotheses existed and CFA provides a more parsimonious model in such cases (Brown, 2006). Therefore, the purpose of CFA is to test or confirm how well a hypothesized factor structure fits or captures the observed data (Kahn, 2006). Comparison of the hypothesized CFA models occurred through goodness-of-fit indices to determine which factor model was the best fit to the data. We followed the standards of Hu and Bentler (1999), who recommend a root mean square error of approximation (RMSEA) of <0.06, and a comparative fit index (CFI) or Tucker-Lewis index (TLI) of >0.95. Additionally, Yu (2002) recommend a weighted root mean square residual (WRMR) of ≤1.0 with binary data.

After CFA confirmed the latent structure, reliability estimates were investigated using the Kuder and Richardson Formula 20 (1937). This formula estimates internal consistency reliability for binary, or dichotomous, items. Commonly, estimates of ≥0.90 are considered excellent, 0.80–0.89 are considered good, 0.70–0.79 are considered fair, and ≤0.70 are considered unacceptable (Cicchetti & Sparrow, 1990).

Finally, measurement invariance analyses followed. Gender is an important variable of interest in understanding depression literacy (Cotton et al., 2006; Olsson & Kennedy, 2010), in particular whether males and females differ on levels of knowledge. Also of interest is whether participants receiving the curriculum from outside health facilitators attain similar rates of depression literacy as those receiving the curriculum from school-based educators. This is an important consideration in the feasibility and sustainability of an intervention. In order to examine these questions, researchers must first determine that the underlying factor structure of a tool measuring this construct is similar for males and females or for those receiving the curriculum from different facilitators. Finally, critical to assessing change pre- to post-test is to establish that the latent structure remains the same over time.

Measurement invariance, or consistency of measurement across some specified groups (e.g., age, sex, race), can be classified as configural, weak, strong, and strict (Meredith, 1993). Measurement invariance is achieved when parameters of the measurement model are not significantly different across groups of interest. In terms of our investigation, we were interested in measurement invariance to the level of weak invariance. In order to evaluate measurement invariance, a series of nested models were fit. First, configural invariance was fit, followed by the constraints set forth by weak measurement invariance. Configural invariance suggests that all parameters (i.e., factor loadings, residual, and intercepts) within the model are allowed to vary across groups. This level of investigation suggests that the underlying factor structure is different between subgroups. Weak invariance forces the factor loadings to be equivalent across groups (such as gender), but allows the residual and intercept parameters to vary by group, suggesting that the underlying factor structure is similar. Fit indices, as described above, were compared to determine the appropriate degree of measurement invariance to impose on the data (Meredith, 1993).

Aim 2: Analyses of the Attitudes Section of the ADKQ

Questions 16–19 (see Table 3) were the focus of these analyses. One author (EAK) reviewed approximately 300 questionnaires and developed a coding schema. This was followed by inter-rater reliability analyses from approximately 90 additional surveys (EAK, KLS, LCG). The coding was conducted independently, and discrepancies were resolved via group discussion until consensus was reached. On the rare occasion consensus was not reached, the final decision was made by the Principal Investigator (KLS). Finally, the categories were explored in the current sample in order to investigate the percentage of participants who endorsed each category using both pre-and post-test data using SPSS version 19 (IBM Corporation, 2010).

Results

Aim 1: Establish the Latent Structure of the Knowledge Section of the ADKQ

CFA

Fit indices are listed in Table 4. For the investigations establishing (Subsample 1) and confirming the latent structure (Subsample 2), all three of the factor models met the established fit indices criteria. Additionally, there was little variation in fit between the factor models. That is, the one-, two-, and three-factor models appeared to fit the data equally well. However, unacceptably high correlations (i.e., r ≥ 0.90) were evident between all factors in both the two- and three-factor models, indicating a lack of distinction between the factors. For these reasons, the one-factor model was chosen and named General Knowledge (KNOW). Factor loadings (Table 2) ranged from 0.52 ≤ λ ≤ 0.98. Finally, while the model did not fit the data as well with pretest responses (Subsample 3), fit indices were still similar across the three models, and again, unacceptably high correlations (r ≥ 0.90) were evident in both the two- and three-factor models. Factor loadings demonstrated a larger range (0.22 ≤ λ ≤ 0.89). Taken together, these analyses provide evidence supporting a one-factor KNOW model of the ADKQ items 1–13.

Table 4.

Fit statistics for the knowledge section of the ADKQ CFA and measurement invariance models

Factors	χ2	DF	CFI	TLI	RMSEA	WRMR
CFA models
Subsample 1 (posttest)
1	261.99**	51	0.97	1.00	0.04	1.40
2	266.17**	51	0.97	1.00	0.04	1.40
3a	256.35**	49	0.97	1.00	0.04	1.38
Subsample 2 (posttest)
1	261.85**	53	0.98	1.00	0.04	1.36
Subsample 3 (pretest)
1	511.68**	57	0.84	0.95	0.05	2.12
2a	509.04**	56	0.84	0.94	0.05	2.12
3	496.53**	54	0.85	0.94	0.05	2.10
Invariance models
Gender	351.71	41	0.99	0.99	0.03	1.48
Facilitator	385.86	41	0.99	0.99	0.04	1.54
Pre- to post-test	1,890.07	310	0.96	0.96	0.04	2.55

CFI Comparative fit index, TLI Tucker-Lewis Index, RMSEA root mean square error of approximation, WRMR Weighted Root Mean Square Residual (only calculated for CFA models) DF degrees of freedom

^**p ≤ 0.01

^aNon-positive definite matrix

Reliability

Internal consistency estimates for KNOW (items 1–13) were good (0.89), indicating a homogeneous test with good reliability.

Measurement Invariance

Fit indices are listed in Table 4. For all analyses (i.e., gender, facilitator, pre- to post-test), fit indices were within the acceptable ranges according to established criteria. For each comparison, weak measurement invariance was indicated, meaning that the model where the factor loadings are equal across group fit the data best. This supports the idea that the one-factor structure holds across groups and across time.

Aim 2: Investigate the Attitudes Section of the ADKQ

First, categories for items 17–19 were developed (item 16 is dichotomous). Table 3 lists categories developed using the 390 participants, along with percentage of participants from the current sample (n = 8,216) who endorsed each category at any point (i.e., either pre- or post-test). For item 17 (identified helper), nine main categories emerged. Percent agreement on coding for this item was 95 % between raters (EAK, KLS, LCG). The most frequently endorsed category by the sample was Family and Mental Health/Medical Professionals. The responses of Parent/Guardian, Counselor, Adult, and Teacher were the most common (i.e., 90 % of all responses).

On item 18 (barriers to help), four main categories emerged. Percent agreement on coding was 94 % between raters. Over one-third to almost one-half of the sample identified at least one issue that would prevent them from accessing help. Only 8.8 % of the sample indicated that there should be nothing to prevent accessing help. Finally, for item 19 (keeping a friend’s depression secret), two categories emerged including telling someone or keeping it a secret. Percent agreement was 87 % between raters.

Discussion

Attitudes and knowledge about mental illness and mental health treatment (i.e., mental health literacy) appear to be important targets for intervention (Fröjd et al., 2007; Olsson & Kennedy, 2010; Watson et al., 2004; Yap et al., 2011). Schools provide a natural context for such universal interventions targeting all adolescents. However, a recent review of the current status of the literature regarding school-based mental health literacy interventions emphasized a number of limitations (Wei et al., 2013). A lack of validated measures was a primary limitation noted. We suggest that addressing the need for a validated measure is a significant contribution to the field. Without a psycho-metrically sound instrument to measure the construct, it is difficult to measure change in the research participants, replications of study findings cannot occur, and researchers have not collectively agreed on an operational definition by which to study the construct.

The current analysis provides an initial examination of the psychometric properties of the ADKQ. The ADKQ is a measure created to assess the effectiveness of ADAP and more generally, to evaluate the construct of depression literacy. To our knowledge, at the time of ADAP and the ADKQ’s development, no measure for the construct of depression literacy existed. Our main findings are as follows: (1) a one-factor (general knowledge) structure of the Knowledge section was established, (2) good reliability of this scale was demonstrated, (3) invariance of the latent structure was confirmed between males and females, school-based and outside health facilitators, and across pre-to post-test, and (4) categories for the Attitudes section were established and explored.

These findings carry important implications. First, establishing one, general knowledge factor reinforces the results of previous research (i.e., Hess et al., 2004; Swartz et al., 2010) demonstrating pre- to post-test improvement in depression literacy after administration of ADAP. These prior studies of the Knowledge portion of the ADKQ used items 1–15 to obtain one score (i.e., percent correct on these items) to assess depression literacy. The current study has confirmed through psychometric analyses that investigating pre- to post-test change in terms of one score for knowledge of mood disorders is appropriate. It is recommended, however, that future studies investigate items 14–15 of the Knowledge section separately, as they are methodologically distinct from items 1–13. Additionally, reliability analyses confirm that the items are unidimensional versus multidimensional.

While we hypothesized that two or three factors would emerge from the factor analyses, there may be several reasons why these hypotheses were not supported. First, as the thirteen items were dichotomous, it may be that the variance needed to parse out differences was limited. Second, as the items were investigating correct responses to questions of a student’s knowledge of depression and bipolar disorder, it may be that the individuals were more likely to know or not know certain information. In other words, that knowledge about mood disorders has an overarching basic foundation. For individuals who had that basic foundation, they knew the majority of the information and therefore the thirteen items “hung” together statistically. Future research should investigate specific item parameters (e.g., item difficulty) in connection with desired outcomes (e.g., help-seeking).

Establishing measurement invariance pre- to post-test also supports investigation of the impact of universal school-based programs, such as ADAP, on depression literacy. As the latent structure remains the same over time, performance can thus be compared. Further, establishing measurement invariance within groups of interest (e.g., sex, facilitator) allows for future analyses to compare these groups. That is, in order to compare across subgroups (e.g., sex) and across time (i.e., pre- to post-test), first measurement invariance must be established. As a result of the current study indicating the latent structure remains the same, further research can actually make these comparisons. For example, as gender appears to be an important variable in depression literacy (Cotton et al., 2006) and help-seeking behaviors (Olsson & Kennedy, 2010; Zwaanswijk, Verhaak, Bensing, van der Ende, & Verhulst, 2003), it will be of interest to see if these universal, school-based curricula have differential effects on males and females. Additionally, sustainability of interventions is a crucial consideration and future research will now allow for investigating differential effects of administration of ADAP by various facilitators. For example, teachers have ongoing access to and relationships with their students, and are able to reinforce lessons in the classroom, which may provide for sustained long-term effects through their prolonged contact (Diekstra, 2008).

Finally, establishing categories for the open-ended questions of the Attitudes section provides opportunities to examine pre- to post-test change in this important, previously un-examined, construct. In addition to increasing knowledge, changing negative attitudes about mental illness and treatment are also important targets for prevention efforts (Fröjd et al., 2007; Watson et al., 2004; Yap et al., 2011). While these constructs are likely linked (i.e., as knowledge increases, negative attitudes decrease; Olsson & Kennedy, 2010), it may also be anticipated that attitudes are more challenging to change than knowledge. Yet, effecting change in this area can be particularly useful. For example, if adoption of a universal, school-based program encouraged more youth to seek help for themselves or others from mental health professionals rather than more informal agents (e.g., peers), opportunities to intervene would increase (Del Mauro & Williams, 2013). Furthermore, understanding the barriers to help-seeking and the impact various interventions have on these barriers will be useful in designing and adapting interventions.

There are several strengths of this study including the large sample size and that the ADKQ addresses a gap in the assessment of adolescent depression literacy with the aim of increasing awareness of early signs and symptoms and treatment for depression among adolescents. These findings must be understood in the context of limitations. Because the project was parental consent exempt, we did not collect other variables that would have been important to study in relation to the ADKQ (e.g., depressive symptoms or mood disorder, ethnicity). We also did not study whether ADKQ results were related to help-seeking or mental health service utilization. However, a randomized-controlled effectiveness study of ADAP is currently underway. This study will utilize the ADKQ to further investigate depression literacy and the impact of this universal school-based program on help-seeking and mental health service utilization. Finally, this study was not designed to capture all possible perceived barriers to help-seeking; therefore, it is possible that additional barriers exist that were not measured in this study.

To conclude, precursors to behavior change include positive attitudes toward the accessing of mental health services and the knowledge, skills, and abilities to use mental health services (i.e., mental health literacy). School health education classes are an ideal forum to enhance depression literacy before the peak age of onset of mood disorders. The current study begins the process of providing psychometric evidence for an assessment tool to measure depression literacy, the ADKQ. Further research is being conducted to demonstrate the utility of the ADKQ, primarily through the investigation of the effectiveness of ADAP. It is anticipated that this measure will benefit the field in two ways. First, researchers may use the ADKQ, a psychometrically evaluated measure, in future studies of depression literacy. Second, school-based mental health professionals may use the ADKQ to screen for the implementation of more targeted interventions with students exhibiting lower levels of literacy. In this burgeoning field of mental health literacy, the ADKQ provides promise.