Overlooked and Underserved: “Action Signs” for Identifying Children With Unmet Mental Health Needs

Abstract

OBJECTIVE:

The US Surgeon General has called for new approaches to close the mental health services gap for the large proportion of US children with significant mental health needs who have not received evaluation or services within the previous 6 to 12 months. In response, investigators sought to develop brief, easily understood, scientifically derived “warning signs” to help parents, teachers, and the lay public to more easily recognize children with unmet mental health needs and bring these children to health care providers' attention for evaluation and possible services.

METHOD:

Analyses of epidemiologic data sets from >6000 children and parents were conducted to (1) determine the frequency of common but severely impairing symptom profiles, (2) examine symptom profile frequencies according to age and gender, (3) evaluate positive predictive values of symptom profiles relative to Diagnostic and Statistical Manual of Mental Disorders diagnoses, and (4) examine whether children with 1 or more symptom profiles receive mental health services.

RESULTS:

Symptom-profile frequencies ranged from 0.5% to 2.0%, and 8% of the children had 1 or more symptom profile. Profiles generated moderate-to-high positive predictive values (52.7%–75.4%) for impairing psychiatric diagnoses, but fewer than 25% of children with 1 or more profiles had received services in the previous 6 months.

CONCLUSIONS:

Scientifically robust symptom profiles that reflect severe but largely untreated mental health problems were identified. Used as “action signs,” these profiles might help increase public awareness about children's mental health needs, facilitate communication and referral for specific children in need of evaluation, and narrow the child mental health services gap.

Despite the presence of well-established diagnostic criteria for children's mental disorders for >20 years,^1–3 parents and teachers often fail to recognize the presence of possible mental health problems in children. Estimates by the US Surgeon General⁴ have revealed that most children with mental disorders do not receive treatment,^5,6 even with well-recognized conditions such as attention-deficit/hyperactivity disorder (ADHD).

Recognition of children's mental health needs depends on the awareness and actions of key adults.^5,6 However, failure to recognize children's mental health needs is not only a problem among laypersons but also among education, welfare, juvenile justice, and health care professionals.^7,8 Underidentification is of particular concern in schools^9–13 and primary care,^4,14–20 where virtually all children are seen and where identification should be feasible. However, only 1 in 4 children with a mental disorder is identified by the primary care doctor.²¹ In fact, the best predictor of a primary care clinician (PCC) identifying a child's mental health problem is whether parents draw the PCC's attention to the issues rather than any PCC-initiated procedure.²² Evidence suggests that PCCs are more comfortable in identifying and managing attention disorders, whereas identification, treatment, and referral for other disorders lag.²³

Communication is complicated by difficulties distinguishing symptomatic from normal behaviors. Symptom lists are often used to help parents and others identify children with mental health needs, but such lists might not be of assistance if the parent, teacher, or physician reasons (correctly) that “every child has some of these symptoms some of the time.” In addition, the time and administrative burdens needed to screen for mental health problems within primary care practices seem to be major deterrents to their use.^23–25 Attempts to remedy this situation have involved computer-assisted²⁵ and symptom-specific²² screening procedures, but these methods also involve time or expense and might be biased toward a single area of functioning (eg, depression screeners).

Better ways to communicate about children's mental health needs must be developed, in terms easily understood by persons of different backgrounds, cultures, and education levels. Ideally, these simpler communication methods should apply the “lessons learned” from studies of decision-making and our understanding of the limitations of human capacities to make complex judgments under uncertainty (eg, to refer or not refer a child for evaluation).^26,27 For example, many cognitive operations are required for parents to (1) determine which, if any, symptoms their child might have from a long list, (2) evaluate the significance of any symptoms in terms of what the parents know about the child, (3) ponder whether the problems are severe enough to bring them to their doctor's attention, and (4) actually schedule a visit. By way of contrast, the cognitive present/not-present determinations of a single well-described criterion, such as a single symptom profile or “warning sign,” greatly simplify the cognitive operations, might reduce uncertainty in decision-making, and facilitate parents and others to obtain a health care evaluation when a child is in distress.

The idea of creating warning signs for health problems is not new. In 1971, President Nixon declared a “war on cancer” with enactment of the National Cancer Act.²⁸ Seven warning signs were developed as a communication tool for early intervention.^26,27 These warning signs were designed to be easily understood so that those with a potential cancer warning sign would realize that a checkup is required. Any single warning sign was meant to trigger an action (ie, seeing one's doctor for an evaluation).

Invoking the need to develop a similar strategy for child mental health, the US Surgeon General^29–31 called for researchers to find more effective means for public communications about the types of children's behaviors that warrant professional attention. This need was further underscored by the President's New Freedom Commission on Mental Health.^32–35

To meet the Surgeon General's challenge, federal officials determined that brief, common-sense descriptions of child mental health problems might be useful if they (1) validly characterize children with mental health problems, (2) are cast in language readily understood by the public, and (3) are accepted by the public as credible indicators of a child's need for a health evaluation. The value of such descriptions would then be for their use as communication/education tools and public messages (ie, as “warning” or “action” signs) to educate and mobilize the public to identify and refer appropriate children for health care evaluations.

Thus, officials from the National Institute of Mental Health and the Center for Mental Health Services supported a contract to empirically determine if certain behavior/symptom profiles might be developed as action signs for public communication and educational objectives. Under federal guidance, we established a steering committee (SC) (that consisted of child mental health epidemiologists, parent/advocacy representatives, and policy experts [see author list]), the goal of which was to develop these warning signs. The SC met regularly (by telephone and in person) to define the goals and methods of the project. The following project objectives were set.

1. To identify brief, easily understood symptom profiles of children with significant behavioral health problems from extant epidemiologic data sets. Because of well-known variations in mental disorder prevalence based on the level of impairment required to meet diagnostic criteria, and given the SC's wish to avoid identifying children without significant problems as having a warning sign (ie, false-positives), the SC focused on symptom profiles for severely ill children for whom a medical evaluation would be deemed medically necessary and noncontroversial, even to those who are skeptical.

2. To determine if these descriptors characterize children with common yet severe behavioral problems and who are not receiving any health care for these problems (eg, unmet need). This second objective was critical, because if the descriptors only identified children who were already receiving health care services, they would not serve the purpose of identifying children with severe yet unrecognized problems. SC members set the goal that the action signs should identify ∼5% to 7% of the community population,³⁵ erring on the side of low sensitivity and high specificity.

3. To ensure that the descriptors map onto recognized, impairing psychiatric diagnoses with positive predictive values of ≥50%.

4. To translate the profiles into common-sense language that can be readily understood by the public and ultimate users.

5. To work with major national professional and advocacy organizations to officially endorse/adopt the warning signs for further dissemination.

Here we present the methods and findings from these 5 goals of our warning-sign development. Please note that, from this point forward, we refer to the warning signs as “action signs” on the basis of recommendations of the SC and multiple focus groups, which indicated that the latter term would be more useful and less likely to be stigmatizing.

METHOD

To identify data sets for which the necessary analyses could be performed, the SC comprehensively reviewed the English-language research literature on epidemiologic studies of childhood mental illness (for children aged 7–17 years), services use, and unmet need by using the Medline and PsycINFO search engines. SC members consulted with the National Institute of Mental Health to find any relevant, federally funded epidemiologic studies not yet in the literature. Ultimately, all federally funded national and regional epidemiologic studies^36–49 were identified, and their results were reviewed. After review of eligible data sets, the SC determined that the final data sets must (1) have used well-validated measures in assessing the range of established psychiatric disorders, (2) have ascertained specific symptoms, symptom patterns, and final diagnoses, (3) provide information about the children's receipt of health care services, (4) have taken place in North America, (5) have been conducted in 1990 or later, and (6) have been drawn from a large, representative community sample.

Data sets from 4 studies were determined to be appropriate and available for analysis: Methods for the Epidemiology of Mental Disorders in Children and Adolescents (MECA)³⁶; Iowa-Georgia Rural Minority Study (Iowa)³⁷; Depression and Anxiety in Minority Youth and Primary Care (Texas)³⁸; and Antisocial Behaviors in US and Island Puerto Rican Youth (Bronx, NY).^39,40 Detailed descriptions of all data sets and their study procedures are available on request. Each data set was representative of the given population within the sampling frame. The sample sizes ranged from nearly 1000 to >4000 children, and child ages ranged between 7 and 17 years. In addition to the necessary characteristics we described earlier, these data sets were chosen because the same diagnostic assessment instrument (the Diagnostic Interview for Children and Adolescents [DISC] versions 2.3 and 4.0^41,42) was used, which facilitated the same analyses across all 4 data sets and enabled us to ultimately combine the data sets for summary analyses.

The SC proposed symptom profiles based on face-valid medical-necessity criteria (ie, that the profile includes a common reason for a child to be seen in clinical settings; that the profile is characterized by substantial impairment or risk for harm; that in most communities the symptom profile would constitute credible reasons among the lay public for seeking a health care evaluation and/or services; and that failing to get an evaluation could put the child at risk for further harm). During SC deliberations a list of symptom profiles quickly emerged; these profiles changed only slightly despite extensive vetting and input from teacher, parent, youth, and PCC focus groups over the ensuing 12 months. All warning-sign constructs included impairment, which entailed 1 or more of the following: (1) extreme emotions that interfere with daily life (eg, severe depression, anger/rage); (2) behaviors that put the child or others at risk for physical harm (eg, suicidal plans/attempts, aggression, purging, or laxative abuse); or (3) extreme inattention or hyperactive behaviors that caused school failure or physical danger. Although some of these problems map loosely onto specific diagnoses, there are many exceptions (eg, the child who is suicidal might actually have substance abuse or panic disorder; the withdrawn child might have a substance use disorder, depression, or schizophrenia). The decision to use symptom/problem profiles rather than diagnoses was based on pragmatic communication and decision-making principles: (1) use common-sense language and constructs familiar to parents, teachers, and youth (peers); (2) avoid jargon or terms that might stigmatize (eg, mental, hyper, schizo, psycho); and (3) keep it simple (ie, avoid symptom profiles that would require people to ponder long Diagnostic and Statistical Manual of Mental Disorders [DSM]^1–3 symptom lists before concluding that a given child has a “problem”).

After input from multiple focus groups, 4 additional constructs were added, namely for repeated use of illegal drugs, extreme problems with attention, mood swings, and drastic personality changes. The latter 2 indicators could not be gleaned from the DISC interviews, because questions pertaining to those constructs are not asked in the DISC.

The DISC was examined to find question items that, if answered positively, would meet the criteria for the proposed initial symptom descriptions. For example, the SC operationalized the depression symptom profile with 3 criteria: (1) depressive symptoms; (2) lasting a minimum of 2 weeks; and (3) resulting in substantial impairment in ≥2 settings (home functioning, school functioning, peer relationships, and/or intrapsychic distress). All of these variables are assessed by the DISC, which makes it possible to define the construct. A complete listing of DISC interview questions that map onto symptom profiles are available from the authors.

To assess impairment, the DISC specifically asks at the end of each section (ADHD, anxiety, depression, etc) whether the symptoms endorsed by the interviewee cause impairment at home, school, or with peers or cause the child severe internal distress. Each impairment item is coded on a 3-point scale (0, none; 1, some; 2, a lot). The SC required that a child have a total impairment score of ≥2, which could mean that the child had “a lot” of impairment at school² but none at home or elsewhere or at least some impairment in ≥2 domains (1 + 1). Likewise, at the end of each section the DISC asks whether the child has seen any mental health, educational, or health care provider in the previous 6 months because of the endorsed problems. Thus, we could determine how many children had a specific symptom profile, whether it caused substantial impairment, whether that symptom profile met full criteria for a DSM-IV diagnosis,³ and whether the child had received any services for the problem(s). If not, by definition that child fell into the mental health services gap definition of having “unmet need.” Frequency analyses according to age and gender were calculated for all indicators and diagnoses.

Parallel analyses were then performed to determine if the findings could be replicated in other data sets. Thus, the Great Smoky Mountains Study⁴⁹ met all of the same criteria as the other 4 data sets except it used the Child and Adolescent Psychiatric Assessment (rather than the DISC) as the diagnostic instrument in this sample of >1000 children. Similar analyses were performed on this sample to determine if findings were specific to the DISC versus more applicable generally. As a final step, after the initial definition and empirical testing of the symptom profiles, the SC convened multiple geographically and ethnically diverse focus groups to (1) solicit feedback about the action signs, (2) ascertain their credibility with parents, teachers, youth, and PCCs, (3) identify any missing action signs, and (4) craft optimal language for laypersons' understanding, use, and eventual dissemination. A description of this focus-group feedback process is available from the authors (P.S.J., unpublished data). The action-sign descriptors initially identified by the SC, as well as the final action signs' wording after the focus groups' input, are shown in Table 1.

TABLE 1.

Final List of Action/Warning Signs

Initial Construct	Final Action Signs After Focus-Group Input
Severe depression with impairment	Feeling very sad or withdrawn for ≥2 wk
Suicidal thoughts with plan or any attempt	Seriously trying to harm or kill yourself, or making plans to do so
Panic attack	Sudden overwhelming fear for no reason, sometimes with a racing heart or fast breathing
Severe aggression with impairment	Involved in multiple fights, using a weapon, or wanting badly to hurt others
Poorly controlled impulses	Severe out-of-control behavior that can hurt yourself or others
Eating disorder with impairment	Not eating, throwing up, or using laxatives to make yourself lose weight
Severe anxiety with impairment	Intense worries or fears that get in the way of your daily activities
Severe inattention/hyperactivity	Extreme difficulty in concentrating or staying still that puts you in physical danger or causes school failure
Substance use	Repeated use of drugs or alcohol
Mood swings	Severe mood swings that cause problems in relationships
Personality changes	Drastic changes in your behavior or personality

RESULTS

Because our ultimate objective was to develop action signs that identify underserved children with significant mental health problems, our first analyses necessarily examined the extent to which the data sets contained substantial numbers of children who met criteria for specific DSM-IV³ diagnoses and who were not receiving services. Disorder-specific prevalence rates and the corresponding proportion of those who did not receive service (shown in parentheses) were as follows, parent versus youth reports, respectively: major depressive disorder, 1.9% (74.5%) vs 1.8% (70.1%); conduct disorder, 1.2% (69.8%) vs 2.2% (88.8%); eating disorder (bulimia or anorexia nervosa), 0.5% (85.7%) vs 0.4% (60.0%); ADHD, 5.4% (63.6%) vs 2.2% (63.1%); substance use disorder, 0.1% (80.%) vs 0.8% (76.5%); and any anxiety disorder, 7.3% (82.5%) vs 8.6% (93.3%). Tables in which disorder rates are compared according to age and gender, with significance values and confidence intervals, are available on request.

However, the action signs were not intended to identify specific diagnoses per se but, rather, were meant to help non–mental health professionals recognize underserved children with significant mental health needs regardless of specific diagnosis to ensure that such children are brought to clinical attention. To avoid action-sign definitions that would identify mildly troubled children and/or those with no clinical disorder whatsoever, the SC stipulated that action signs should be associated with substantial impairment, such as that required for any DSM diagnosis. However, a child might exhibit an action sign and have a quite different diagnosis, which makes it difficult to determine if any specific action sign is linked to impairment. To address this problem, we computed an overall “any-diagnosis” category to assess how well the presence of 1 or more action signs mapped on to the any-diagnosis criterion (and, by extension, clinically significant impairment). Any-diagnosis prevalence rates for children are listed in Table 2, separated according to age and gender groups. As seen here, regardless of whether diagnoses were determined by child or parent reports, ∼15% met criteria for 1 or more DSM-IV disorders,³ which is consistent with the results of most previous studies^36–49 but slightly higher than those of a recent epidemiologic study.⁵²

TABLE 2.

Number and Proportion of Children With Any Diagnosis According to the DISC

Youth sample (N = 7604)a
Total, n/N (% [95% CI])	1152/7604 (15.2 [8.6–24.3])
Gender
χ2	0.43
df	1
P	NS
Age
χ2	14.8
df	1
P	<.0001
Age 7–11 y, n/N (% [95% CI])
Male	152/1235 (12.3 [8.9–21.7])
Female	153/1146 (13.4 [7.3–20.3])
Age 12–17 y, n/N (% [95% CI])
Male	448/2658 (16.8 [6.9–28.8])
Female	399/2565 (15.6 [9.1–23.6])
Parent sample (N = 4621)b
Total, n/N (% [95% CI])	733/4621 (15.2 [12.1–19.4])
Gender, n/N (% [95% CI])
χ2	0.13
df	1
P	NS
Age, n/N (% [95% CI])
χ2	0.84
df	1
P	NS
Age 7–11 y, n/N (% [95% CI])
Male	277/1751 (15.8 [12.6–19.8)
Female	205/1353 (15.2 [10.2–20.7)
Age 12–17 y, n/N (% [95% CI])
Male	122/735 (16.6 [7.7–24.7)
Female	129/779 (16.6 [7.7–20.9)

df indicates degrees of freedom; NS, not significant.

^aSample size was 7604, which the combines MECA, Texas, Iowa, Bronx, and Puerto Rico samples.

^bSample size was 4621, which combines the MECA, Iowa, Bronx, and Puerto Rico samples.

Table 3 indicates the number and proportion of children with specific action signs, as well as those who met the any-action-sign criterion (ie, ≥1), which yielded <10%, parents or youth. Thus, as intended, the indicators identified a smaller proportion of children (presumably those more severely affected) than the larger proportion that met DSM-IV diagnostic criteria. Also seen here is that high levels of unmet need were found among children with 1 or more action signs that ranged from 60% for depression to 88.9% for aggression (specific findings of levels of unmet need for each action sign are available on request).

TABLE 3.

Action-Sign Frequency According to Age, Gender, and Informant

Symptom Profile	Youth Samplea			Frequency Comparison			Parent Samplec
	Age 7–11 y, %	Age 12–17 y, %	Total, n/N (%)	χ2	dfb	P	Age 7–11 y, %	Age 12–17 y, %	Total, n/N (%)
Any indicator
Male	6.1	10.4	753/7604 (9.9)	36.9	1	<.0001	5.4	8.8	310/4621 (6.7)
Female	8.3	12.0	—	—	—	—	6.0	8.7	—
Severe sadness
Male	0.4	1.1	84/7604 (1.1)	0.74	1	NS	0.7	1.9	59/4621 (1.3)
Female	0.6	1.6	—	—	—	—	1.2	2.0	—
Suicidality
Male	1.9	2.7	238/7604 (3.1)	75.7	1	<.0001	0.6	1.4	34/4621 (0.7)
Female	1.8	4.7	—	—	—	—	0.4	1.0	—
Severe anxiety
Male	1.9	2.4	180/7604 (2.4)	8.1	1	<.005	2.7	4.2	149/4621 (3.2)
Female	3.2	2.2	—	—	—	—	2.8	4.2	—
Vomiting or laxative use
Male	0.1	0.3	30/7604 (0.4)	18.3	1	<.0001	0.0	0.0	0/4621 (0.0)
Female	0.3	0.7	—	—	—	—	0.0	0.0	—
Fighting/weapon use
Male	1.0	2.4	140/7604 (1.8)	22.9	1	<.0001	0.7	0.8	36/4621 (0.8)
Female	1.2	2.0	—	—	—	—	0.6	1.2	—
Dangerously inattentive or hyperactive
Male	0.4	2.4	128/7604 (1.7)	3.5	1	<.06	1.1	1.9	58/4621 (1.3)
Female	1.6	1.6		—	—	—	1.4	0.8
Repeated drug use
Male	0.2	1.1	65/7604 (0.8)	27.4	1	<.0001	0.0	0.4	5/4621 (0.1)
Female	0.2	1.2		—	—	—	0.0	0.3

^aSample size was 7604, which combines the MECA, Texas, Iowa, Bronx, and Puerto Rico samples.

^bχ² statistics refer to frequency comparisons between youth and parent informants irrespective of gender and age groups.

^cSample size was 4621, which combines the MECA, Iowa, Bronx, and Puerto Rico samples.

Major differences in frequencies can be seen between parent and youth reports in Table 3. One extreme example is the difference in recognition of youths' suicidal behavior by youth and parents: using the youth report as the gold standard, parents are only aware of approximately one-third of all youth with suicidal plans and one-sixth of youth with actual attempts. Parents are also less aware of eating-behavior problems (which might be expected given the secrecy of the disorder), repeated drug use, and severe aggression or carrying a weapon. These differences in the informant-described prevalence of the action signs are generally consistent with results from previous studies.^50,51

Table 4 shows the extent to which subjects who met the any-action-sign criterion also meet the any-diagnosis criterion. These data reveal moderate-to-high positive predictive values (52.7% youth report, 74.8% parent report) of action signs to DSM-IV diagnoses, which indicates that children with 1 or more action signs have clinically significant, impairing problems.

TABLE 4.

Positive Predictive Value of Having 1 or More Action Signs (Any Action Sign) Versus Any DSM Diagnosis

Any Diagnosis/Any Indicator	Youth Sample, n/N (%)a	Parent Sample, n/N (%)b
Total	398/755 (52.7)	231/309 (74.8)
Age 7–11 y
Male	34/75 (45.3)	73/95 (76.8)
Female	49/95 (51.6)	57/81 (70.4)
Age 12–17 y
Male	158/277 (57.0)	51/65 (78.5)
Female	157/308 (51.0)	50/68 (73.5)

^aSample size was 7604, which combines the MECA, Texas, Iowa, Bronx, and Puerto Rico samples.

^bSample size was 4621, which combines the MECA, Iowa, Bronx, and Puerto Rico samples.

In terms of unmet need, Table 5 lists the number and proportion of children in services with any diagnosis (≥1) or any action sign (≥1). For children who met the any-diagnosis criterion, most were not receiving services in the previous 6 months (75.2%–86.9%). Likewise, despite the likely greater severity of the action-sign profiles, 73.8% (parent DISC) and 83.2% (youth DISC) of children who met any action-sign criteria received no health care assessment or services during the previous 6 months.

TABLE 5.

Unmet Needs: Number and Percentage of Children Who Did not Receive Services, According to Any Diagnosis or Any Action Sign

	No Services/Any Diagnosis, n/N (% [95% CI])	No Services/Any Indicator, n/N (% [95% CI])
Youth samplea
Total not receiving services	1001/1152 (86.9 [85.2–93.0])	628/755 (83.2 [82.0–86.2])
Age 7–11 y
Male	137/152 (90.1 [85.2–93.5])	62/75 (82.7 [50.0–92.3])
Female	136/153 (89.5 [84.7–93.6])	78/95 (82.1 [51.1–87.5])
Age 12–17 y
Male	383/448 (85.5 [72.7–87.6])	229/277 (82.7 [75.0–100.0])
Female	344/399 (86.2 [84.0–93.2])	259/308 (84.1 [81.6–100.0])
Parent sampleb
Total not receiving services	551/733 (75.2 [68.6–83.4])	228/309 (73.8 [71.2-82.9])
Age 7–11 y
Male	213/277 (76.9 [61.3–85.7])	76/95 (80.0 [63.6–93.3])
Female	90/122 (73.8 [56.2–100.0])	47/65 (72.3 [62.5–100.0])
Age 12–17 y
Male	157/205 (76.6 [56.2–88.6])	57/81 (70.4 [53.6–94.4])
Female	91/129 (70.5 [46.7–100.0])	48/68 (70.6 [66.7–76.9])

^aSample size was 7604, which combines the MECA, Texas, Iowa, Bronx, and Puerto Rico samples.

^bSample size was 4621, which combines the MECA, Iowa, Bronx, and Puerto Rico samples.

The parallel analyses with the Child and Adolescent Psychiatric Assessment instrument reinforced these findings. In this sample, for which a different diagnostic instrument was used, approximately two-thirds of the children (63.6%) with an indicator had not seen anyone for evaluation or services before the interview period. Also similar to our findings earlier, analyses that compared the indicators with actual Child and Adolescent Psychiatric Assessment diagnoses revealed moderate-to-high positive predictive values (range: 30.6%–100%; average: 45.8%) (tables are available on request). The final action signs and toolkit prepared for the US Department of Health and Human Services Center for Mental Health Services is available online at www.thereachinstitute.org/files/documents/action-signs-toolkit-final.pdf.

DISCUSSION

According to these analyses, the overall prevalence of mental health disorders among children and adolescents is ∼15%, whether by parent or youth report (note that not all diagnoses were included in these samples [eg, oppositional defiant disorder, Tourette syndrome, autism, etc], so rates including these conditions would likely be higher, perhaps at the generally acknowledged levels of 20%).²⁸ In contrast, the prevalence of more severe indicators ranged from 6.7% for parent reports to 9.9% for youth reports, which yields rates that more closely approximate the efforts by the Center for Mental Health Services to identify the prevalence of “severe emotional disorder.” However, the high levels of children with significant behavioral or mental health problems who were not receiving any services in the previous 6 months are alarming, particularly because the action signs were intended to identify those with an undisputed (face-valid) medical necessity for evaluation and possible services.

These statistics reveal a sobering problem: among children with severe types of problems of greatest concern to all focus groups, most are not being seen by any type of provider, neither health care nor educational. These findings, sadly, are not new. Even for the ADHD indicator, a presumably overdiagnosed disorder, of the 1% of children who had hyperactivity so severe that it put them at physical risk or resulted in school failure, only one-third had received an assessment or had been in care in the previous 6 months. Most dramatically, for action signs related to purported abnormal behaviors such as aggression, children are almost never in services, as shown by our data. Thus, among children who are viewed by stakeholders as having severe types of behavioral health problems, lack of services was extremely high, which is an important new but uniform finding across all 4 of the methodologically rigorous data sets.

The credibility and face validity of the proposed action signs was apparent in the analyses of action signs according to diagnosis (Table 5). On the whole, the action signs mapped onto specific DSM diagnoses 52% to 75% (positive predictive value) of the time and onto the reality that the overwhelming majority of these children do not receive any health care services for their problems. The replication of analyses with the Great Smoky Mountains Study revealed the validity of our findings independent of the DISC instrument.

By way of caution, it should be noted that, by design, the action signs have low sensitivity to most diagnoses, which is easily seen when one views Table 3 and the prevalence rates for any disorder (15.2% and 15.2%, parent and child report, respectively) and compares them to the lower rates for any action sign in Table 4 (6.7% and 9.9%, parent and child report, respectively). Thus, the sensitivities of the any-action-sign metric against the any-disorder variable is low (0.31 and 0.34, parent and child, respectively), whereas the specificities are high (0.98 and 0.95, respectively). As a consequence, if one were to use the action signs as some type of screener, two-thirds of the children with DSM disorders would be missed. On the other hand, the action signs might be useful for educational and broad public service messaging, because they were deliberately designed to avoid risk for harm by overidentifying children or alarming parents. It is important to note that despite their low sensitivities, 70% to 80% of the children the action signs do identify have not received relevant services in the previous 6 months, which indicates that the action signs do seem to effectively identify severely ill children, most of whom have unmet need.

Overall, the action signs identified approximately equal numbers of boys and girls, although specific action-sign rates varied according to gender (eg, substantially greater rates for the anxiety-problem action signs in 7- to 11-year-old girls than boys or the twofold-greater rates of extreme hyperactivity in 12- to 17-year-old boys than girls). It should be noted that the prevalence of action signs seemed to increase with age (from 6% to 8% in the younger age groups to 8% to 12% in older children and adolescents). The difference might reflect a real variation in rates of severe, action-sign–type problems in children versus adolescents, or it might signify a need to identify profiles that better identify younger children. Unfortunately, among children younger than 7 years, we are unaware of any representative data in the United States that could be used to develop and test action signs in this age group, so we could not address this potential problem in this report.

Clinical Implications

This project has broad clinical, service, and policy applications. Scientifically valid, easily understood, and readily accessible action signs might facilitate children's pathways to referral and receipt of needed services and warrant further study and application if we are to close the mental health services gap for children with severe emotional and behavioral difficulties.^53,54 Although new, effective treatments are increasingly available, such treatments will not reach children in need without informed parents and well-educated primary care providers, teachers, and others who are armed with efficient and easily applied means of identifying which children need these services.

Limitations

Although the action signs might serve as effective educational and communication tools for facilitating awareness among parents, teachers, youth, and health care professionals about children's mental health needs, additional studies are needed to ensure that parents and teachers understand them; to determine if they “take action” in appropriate fashion when a child manifests an action sign; and that children and families are bridged effectively over the mental health services gap. Additional studies should also conduct independent evaluations of impairment associated with the various action signs to firmly establish whether they do, in fact, identify more severely impaired children.

We noted that children and adolescents might have other problems and/or significant impairment in functioning not adequately defined by these action signs. For example, should “cutting” be included in some future action signs? Likewise, epidemiologic data sets in which symptoms in very young children were assessed were not available for our analysis, which resulted in significant gaps in our understanding about appropriate action signs in this younger age group. In addition, the full spectrum of diagnoses (autism, Tourette syndrome, enuresis/encopresis, milder cases of ADHD, oppositional defiant disorder, etc) were not included or captured by the current action signs, in part because the DISC does not address conditions such as autism and in part because the SC chose to focus on common and severely impairing conditions.

Finally, we noted that a number of important variables (such as ethnicity, family income) were not available from all 4 data sets, so we could not determine if these variables were related to different levels of action signs. Further analyses of other data sets with a broader array of potential moderating variables could yield additional information about how such factors might affect the distribution of action signs across different population groups. Available evidence suggests that these variables can exert significant effects, particularly in relation to access to and utilization of care and, by extension, unmet needs for services.^54,55

Despite these caveats, the action signs reported here do seem to characterize children across major gender and age groups who have relatively common yet credible behavioral health problems that, more often than not, are not being addressed. Applying action signs to a variety of settings could have a significant impact on early identification and access to services for many underidentified children. However, more research is needed to ensure the ultimate usefulness and broad-scale applicability of the action signs and to determine if they can serve as effective tools for educating the public to better recognize and respond to children's needs for behavioral health services. If appropriately coupled with other efforts to actually improve our diagnostic and treatment services for these children via guidelines and toolkits, substantial progress in children's behavior health outcomes might soon be possible.

The action signs and an associated toolkit are available at www.thereachinstitute.org/files/documents/action-signs-toolkit-final.pdf. The final federal report is available from the corresponding author on request.