An Alternative Technique for Youth Risk Surveillance Outside of the School System

Abstract

Purpose:

When school districts choose not to participate in adolescent health behavior surveys, tracking adolescent health indicators can be challenging. We conducted a countywide youth behavior survey outside of the school system. Our purpose is to describe alternative methods used for gathering these data reliably and ethically.

Methods:

We implemented two parallel surveys with youth ages 14–19 residing in a mid-sized county with urban, suburban, and rural neighborhoods. An anonymous phone-based survey used computer-assisted telephone interviewing with a live interviewer in conjunction with an interactive voice response system to survey youth via random digit dialing of landlines and cell phones. A concurrent in-person anonymous survey was conducted with marginalized youth (from juvenile detention centers, shelters, and residential facilities), using audio computer-assisted self-interviewing technology. The survey measures included the Centers for Disease Control Youth Risk Behavior Surveillance System and additional questions about social supports, neighborhood, and adverse childhood experiences.

Results:

Data were collected between February and December 2014. The phone-based sample recruited 1813 participants; the marginalized sample included 262 youth. Several strategies ensured anonymity and reduced coercion. The final phone-based sample was similar to demographics of the county population. The marginalized youth sample captured out-of-home youth who may have been missed with phone-based sampling alone.

Conclusions:

We review alternative strategies for obtaining population-based adolescent health data without the cooperation of schools. These techniques can provide a basis to collect data that may help direct resources and policies relevant to needs of local youth.

The Centers for Disease Control Youth Risk Behavior Surveillance System (YRBSS), optionally administered biannually across U.S. high schools, provides surveillance data about key adolescent health indicators [1,2]. These data help guide public health practitioners and policymakers to develop youth-relevant policies to influence youth morbidity and mortality [1,3–5]. Some jurisdictions may lack the financial or political support to administer the YRBSS in whole districts to generate local data, necessitating alternative approaches to collect similar data. Even when schools do participate in population-based surveys, youth with chronic absenteeism, those who have dropped out of school (legal at the age of 16 in most countries), are marginally housed, incarcerated, or in residential placement are missing from these samples. In some countries, girls with limited school access may also be disproportionally underrepresented. Thus, surveys administered in schools may miss youth with significant health and social challenges.

None of Allegheny County’s 42 school districts have conducted the YRBSS with their entire student population. While a few have participated in statewide sampling, large districts like Pittsburgh Public School District have not; citing concerns about sensitive questions and time required. As statewide sampling is insufficient for county-level analysis, the lack of county-specific data limits analysis of local youth risk behaviors. Local health departments need county-level data to efficiently target interventions. To address this issue, multiple stakeholders came together to identify innovative methods for collecting YRBSS data at the county level outside of schools.

This project sought to address barriers to collecting comprehensive adolescent health behavior data. The primary aim was to implement a survey similar to the YRBSS with a representative sample of county youth using random digit dialing (RDD) [2]. A secondary aim was to administer a parallel survey with “marginalized youth” who were likely to have more poor health indicators and would not be easily reached using RDD. The purpose of this paper is to describe the methods we used to conduct a countywide survey similar to the YRBSS, independent of schools.

Stakeholder Engagement

The Division of Adolescent and Young Adult Medicine at Children’s Hospital of Pittsburgh of the University of Pittsburgh Medical Center, the Institute for Evaluation Science in Community Health at the University of Pittsburgh Graduate School of Public Health, and the Allegheny County Health Department collaborated to conduct a one-time, cross-sectional, anonymous survey of English-speaking youth in the county. The survey team presented the data collection plan to local philanthropic foundations and discussed the benefits of such data to guide local adolescent health policy and programming. Four local foundations came together to provide funds to conduct the surveys.

The local funders, health department leaders, youth-serving community partners, and research team worked together to finalize the survey. Stakeholder input guided inclusion of additional validated survey questions about adverse childhood experiences, gender and sexual identity, social supports, and neighborhood connectedness not routinely included in the YRBSS. Community partners (youth-serving agencies mostly affiliated with the county’s Department of Human Services) also assisted the research team in recruiting for the marginalized youth sample.

Methods

Sampling

To obtain a countywide sample of high-school aged youth (ages 14–19) comparable to the YRBSS, we used an RDD methodology similar to what is generally used for the Behavioral Risk Factor Surveillance Survey [6]. Allegheny County is a moderately sized county in western Pennsylvania, with approximately 94,690 youth. We estimated that 1,600 observations would be needed for a 2–3 percent margin of error using a 95% confidence interval. A sample of this size would also allow us to effectively examine differences in health risk indicators. A parallel, in-person survey conducted with a convenience sample of youth made up the “marginalized youth sample.” To capture varied experiences, we used a wide convenience sampling strategy of youth who are marginally housed, incarcerated, or in residential homes based on county human services estimates. The marginalized youth sample included 262 youth, about 10% of the marginalized youth population.

Measures

Demographics and health behavior questions were all validated measures from the Centers for Disease Control YRBSS with additional questions about nutrition and physical activity adapted from the Behavioral Risk Factor Surveillance Survey [2,7]. Participants were also asked about exposure to violence and neglect using previously validated items from the National Survey of Children’s Exposure to Violence, as well as questions about hope and future orientation, social supports, and neighborhood cohesion [8–11]. There were 145 items on this 30-minute survey.

Procedures for Phone-Based Sample

We recruited youth via RDD with two separate frames, one for landlines and another for cell phones. A professional survey center provided intensive training and personnel to interview for the phone-based surveys. Training included interviewing skills, refusal conversion, dispositioning of call attempts, and rehearsal of interview scripts. We oversampled landlines as preparatory work, which showed greater likelihood of finding homes with eligible youth with landline numbers.

Youth provided verbal assent (for minors) or consent (for those ages 18 or 19). The University of Pittsburgh Institutional Review Board (IRB) approved a waiver of written documentation of consent and a waiver of parental permission. Waiver of written consent was permitted because the IRB understood that the study was completely anonymous and less than minimal risk, that questions in the YRBS are common in a confidential clinical encounter, and that the youths were capable of providing assent to answer questions about their own behaviors. We asked 1,860 teens if they wanted us to speak with their guardians, with 169 (9.1%) stating they wanted parental approval. A total of 1,682 (90.4%) did not want parental approval and 9 (.5%) refused to participate. Of the 169 requesting we speak to their parents, for 93.4% of parents agreed, 3.7% refused, and 2.9% of the youth hung up waiting for a parent to answer. Request for waiver of parental permission followed guidance outlined by Olds (2003) and Diviak (2004) and in both surveys included components required by the U.S. Federal Research Policy; justification of minimal risk as it was an anonymous survey, protection of youth rights through privacy and noncoercion, poor feasibility of study without the waiver, and the provision of additional information via resource sheets and phone numbers [12,13]. Justification in the marginalized sample was the same, though we noted the possible lack of parental availability. The IRB also recognized that requiring parental consent could marginalize youth without a readily available adult caregiver to provide consent for participation.

We gathered survey data using computer-assisted telephone interviewing (CATI; WinCati version 4.2; Sawtooth Technologies, Northbrook, IL) with a live interviewer then switched to an interactive voice response (IVR) system for sensitive topics such as sexual identity, drug use, etc. (Marketing Systems Group; ARCS v.6.5 a, Horsham, PA). Live interviewers explained the IVR system to the participant prior to switching over from CATI to IVR. IVR asks previously recorded questions’ answers via phone keypad. Both CATI and IVR automatically save responses. A separate log was created to track and call respondents who completed CATI but not IVR.

Participants were directed to a separate voice mailbox where they left contact information to mail them a $20 gift card and list of youth-relevant county resources. The contact information was never connected to survey data. The phone survey was conducted over a 10-month period from February 2014 to November 2014.

Procedures for Marginalized Youth Sample

The term “marginalized” intends to underscore structural factors precluding youth from participating in an RDD survey. We engaged six local community-based youth-serving agencies (e.g., foster care agencies, transitional housing/residential treatment facilities, detention centers, homeless youth shelters) to facilitate access to this group of youth from 18 different countywide sites. The research team employed a participatory evaluation approach to guide recruitment and ensure confidentiality, privacy, and autonomy. Data collection occurred over 5 months from August to December 2014. Youth in this marginalized sample were not interviewed by phone but completed surveys “in-person” on a laptop via audio computer-assisted self-interview technology with headphones (to address possible low literacy and privacy). Youth often move from one system or institution to other placements, so to reduce the likelihood of youth taking the survey twice, a consistent group of research assistants collected data across sites. Research assistants were experienced with survey methods and trained by authors H.A. and E.M.

For youth who were being served at community sites (i.e., not incarcerated), community partners were provided with recruitment materials to share with interested youth. Community partners were asked to emphasize that participation was completely voluntary and that staff would be blind to youth participation. Research staff arrived to community sites at predetermined dates and times and youth expressing interest participated.

Incarcerated and detained youth were selected at random from a computer-generated number sequence created on each day of survey administration. Facility staff called youth to the medical office at the detention center. Participants were verbally informed that the survey is voluntary and participation would not result in any penalties or incentives. To mask the youth’s decision to participate from detention center staff, if youth did not want to participate, they were given the option of reading an institutionally approved magazine in the privacy of the exam room for 10 minutes as if they were completing the survey.

The IRB reviewed this protocol in full committee with a designated prisoner representative to ensure that study procedures minimized coercion and mitigated risk. Parental permission and written documentation of consent were waived. Community-recruited youth received $20 for their time directly after survey completion. Following juvenile detention center policy, youth recruited in this venue did not receive any monetary incentive. Given the anonymity of the survey and the automation of sensitive questions, research staff were not notified of answers that would trigger mandatory or crisis reporting. However, a list of youth-relevant county resources was provided to all participants.

Data Management

The phone survey data (except for zip code and school district which could inadvertently identify a participant) once cleaned and weighted against the Allegheny County census were made available for public use through the Allegheny County Health Department (http://www.achd.net/hats/). Survey data were weighted for design effects including weights for probability of selection, number of eligible teens in each household, and for the number of landline and cell phones each adolescent had access to. Iterative proportional fitting was applied to adjust for nonresponse by adjusting design weights across several dimensions to ensure weighted frequencies matched population totals obtained from Census 2010 SF1 Public Use Microdata Sample and other external sources. The marginalized youth survey data are separate and private, available by request only, and housed in a secure, password-protected database within the Division of Adolescent and Young Adult Medicine. Findings will be made available via topic-specific briefs for key stakeholders, including youth-serving agencies, policymakers, and local school boards with the goal of increasing interest and support in conducting school-based YRBSS in the future.

Analysis

This manuscript focuses on the feasibility of administering a population-based countywide survey of adolescent health behaviors. We present participation rates as well as some descriptive analyses qualitatively comparing some demographic characteristics between the phone survey and the marginalized youth sample. Demographic variables are presented as unweighted frequencies and percentages, as our purpose is to compare the quality of the samples obtained and not to produce or report on county estimates of health and health-related indicators.

Results

Response rates

The phone-based sample included 1,813 youth; of those 1,609 (88.8%) completed the entire survey. Interviewers called selected numbers at least three times and up to 14 times until a permanent disposition was assigned. With landlines and cell phone sampling frames combined response rates were 38.4% and cooperation rates were about 50%. After switching to IVR,1,554 of the 1,645 who started the IVR portion completed the survey, a loss of 9%. Similarly, of the 1813 who started the survey, 1,645 completed the interviewer portion, also a loss of 9% suggesting that the modes are similar in attrition rates. Table 1 details samples by disposition and frame. Calls took place between 1 P.M. and 9 P.M. weekdays, and 11 A.M. to 6 P.M. weekends. For follow-up calls, to increase efficiency, staff gave preference to publicly listed numbers during the hours when youth were more likely to be at home (evenings and weekends) and to unlisted numbers and cell phones during weekday hours. The survey team called approximately 38,000 telephone numbers in the phone-based sample. Five percent of the numbers called, resulted in surveys being completed.

Table 1.

Healthy Allegheny Teen Survey: Allegheny County, Pennsylvania: Total calls for RDD sample

Final disposition codes	Total number of dispositions
	Cell phone	Landline	Combined
Partial or total survey completion	379	1,434	1,813
Eligible but not interviewed	1,177	2,461	3,638
Unknown eligibility	9,725	11,194	20,919
Confirmed ineligible	14,798	19,745	34,543
Totals	26,079	34,834	60,913
Rates
AAPOR response rate 4	20.02%	38.74%	32.20%
Cooperation rate	32%	58%	50%

AAPOR = American Association of Public Opinion Research; RDD = random digit dialing.

There were 262 participants in the marginalized youth sample, a nonprobability convenience sample, with 95% of the 273 eligible youth approached completing the survey. There are an estimated 3,000 youth in the region being served in the agencies where recruitment occurred [14,15].

Demographics

Table 2 summarizes demographics in the phone-based sample. There was an even distribution of sex and age. The phone-based method captured 78 youth who were out of school (4.3%). About 48% of youth who reported not being in school were high school age (14–18). The phone-based sample closely mirrored the demographics of the county based on all age census data (Table 2) [16]. Most youth had guardians who had at least a high-school education; 67.5% reporting post high-school education. About five percent of youth were identified as sexual minority (ranging from not completely or mostly heterosexual). The phone-based sample revealed few gender nonconforming participants (n = 11 or .7%). Slightly less than 10% of youth from the phone-based sample, (9.2%) reported physical, mental, or emotional limitation. Youth in the phone-based sample were predominantly stably housed with 4.2% reporting past-year homelessness.

Table 2.

Healthy Allegheny Teen Survey: Allegheny County, Pennsylvania: Demographics illustrating effectiveness of survey techniques for random phone-based sample^a

Variable	Phone-based sample		Allegheny County 2010 population totalsb
	N	Percentage (CI) (N = 1,813)	N	Percentage (N = 94,690)
Age
14–15 years old	637	35.1 (32.9, 37.3)	28,500	30.1
16–17 years old	704	38.8 (36.6, 41.1)	30,490	32.3
18–19 years old	472	26.0 (24.0, 28.1)	35,700	37.7
Sex
Male	899	49.5 (47.3, 51.9)	48,340	51
Female	913	50.3 (48.1, 52,7)	46,350	49
Intersex/other	1	0.1	—	—
Race/ethnicity
Non-Hispanic white	1,429	79.7 (77.9, 81.6)	68,900	72.8
Non-Hispanic black/otherc	311	17.4 (15.6, 19.1)	23,670	25
Non-Hispanic black	181	10.1	—	—
Non-Hispanic other	130	7.3	—	—
Hispanic/Latino	50	2.90 (2.1, 3.7)	2,120	2.2
Highest grade guardian completed
Some high school or less	369	25.1
Graduated high school	110	7.5
Some college or technical school	581	39.5
Finished college	412	28
Physically/mentally/emotionally limited
No	1,630	90.8
Yes	165	9.2
Past-year homelessness (2+ nights)
No	1,717	95.8
Yes	76	4.2
Sexuality
Mostly/completely heterosexual	1,480	94.8
Bisexual	49	3.1
Mostly/completely homosexual	33	2.1
Gender identity
Masculine	782	49
Feminine	757	47.4
Androgynous/genderqueer	11	0.7
Other/do not know	46	2.9
In school
No	78	4.3
Yes	1,728	95.7
Grade
Less than 10th grade	512	29.6
10th–11th	690	39.9
12th	263	15.2
Finished or active pursuit of GED	58	3.4
College	204	11.8

^aExcludes unknown, missing, and refused responses.

^bEstimates based on Census 2010 SF1 Public Use Microdata Sample.

^cCombined to compare with Census data.

Table 3 shows results of marginalized youth sample. The marginalized youth sample was majority (60%) male. One-fifth of the marginalized youth were not in high school or college; 20% of those not in schools were high school aged. Of the 18- and 19-year olds in the marginalized youth sample, 38 were not in school and 10 (26%) lacked a high school diploma or GED. Two-thirds of the marginalized youth sample were non-Hispanic African-American, disproportionately higher than countywide demographics. Only 39.3% of youth from marginalized sample had parents with more than a high-school education. About one in five marginalized youth were identified as sexual minority at 21.7%. The marginalized youth sample had a significant number of those identifying as gender nonconforming (androgynous or gender queer; n = 20 or 8.0%). Over a quarter (26.3%) of the participants from the marginalized sample reported physical mental or emotional limitations. Notably, two of every five of the marginalized youth (40.2%) reported any past-year homelessness greater than two nights.

Table 3.

Healthy Allegheny Teen Survey: Allegheny County, Pennsylvania: Demographics illustrating effectiveness of survey techniques for marginalized youth convenience sample^a

Variable	Marginalized youth sample		Allegheny County 2010 population totalsb
	N	Percentage (N = 262)	N	Percentage (N = 94,690)
Age
14–15 years old	77	29.6	28,500	30.1
16–17 years old	96	36.9	30,490	32.3
18–19 years old	87	33.5	35,700	37.7
Sex
Male	152	58	48,340	51
Female	109	41.6	46,350	49
Intersex/other	1	0.4	—	—
Race/ethnicity
Non-Hispanic white	41	15.8	68,900	72.8
Non-Hispanic black/otherc	202	77.9	23,670	25
Non-Hispanic black	168	64.8	—	—
Non-Hispanic other	34	13.1	—	—
Hispanic/Latino	16	6.3	2,120	2.2
Highest grade guardian completed
Some high school or less	76	30.2
Graduated high school	77	30.6
Some college or technical school	38	15.1
Finished college	61	24.2
Physically/mentally/emotionally limited
No	191	73.7
Yes	68	26.3
Past-year homelessness (2+ nights)
No	151	57.8
Yes	105	40.2
Sexuality
Mostly/completely heterosexual	177	78.3
Bisexual	33	14.6
Mostly/completely homosexual	16	7.1
Gender identity
Masculine	114	48.1
Feminine	73	30.8
Androgynous/genderqueer	20	8.4
Other/Do not know	30	12.7
In school
No	49	19.1
Yes	209	79.8
Grade
Less than 10th grade	61	29.2
10th–11th	72	34.4
12th	32	15.3
Finished or active pursuit of GED	30	14.4
College	14	6.7

^aExcludes unknown, missing, and refused responses.

^bEstimates based on Census 2010 SF1 Public Use Microdata Sample.

^cCombined to compare with Census data.

Discussion

Without school district support for in-classroom administration, youth health behavior surveillance at a county level is challenging. We successfully conducted a countywide, population-based youth health behavior survey outside of the school system—the first countywide phone-based survey for youth. RDD methods achieved a relatively accurate reflection of local demographics. The intent was to guide public health policy and practice related to adolescent health while increasing the visibility of adolescents in the county and to allow youth-serving agencies to better understand specific needs and strengths of the youth they serve. In fact, the results have begun to inform work and policy in a variety of areas including an assessment of the state of minority girls in the county.

We effectively demonstrated an alternative technique to collecting this vitally important health behavior data. Even when done in schools, these data are mostly collected by sampled classrooms, or school districts, and whole counties are rarely involved, as has been the case with Allegheny County [1]. To our knowledge, the only prior attempt to conduct a phone survey about adolescent health behavior was conducted by the Pennsylvania Department of Health using an already recruited household panel that required parental consent [14]. Our survey was unique in its combined use of CATI and IVR technologies, waiver of parental permission, and use of RDD rather than an established panel of survey participants. The approach to conducting a countywide adolescent health survey presented here provides alternatives for counties where school districts do not participate in youth health behavior surveys as well as those counties interested in expanding beyond school district level data.

Stakeholder input facilitated the addition of a parallel survey with marginalized youth. The marginalized youth sample was intended to be a heterogeneous group of youth who would likely be missed in a phone survey, with the final sample size determined by resources available to recruit youth from these multiple diverse settings to participate in this survey [17]. Additionally, previous studies of Pittsburgh youth in the populations we sample show similar demographics supporting the idea that we achieved a sampling of that group that was meaningful [15]. Surveying youth who would be otherwise missed in a school-based or phone-based survey helps ensure that the needs of this vulnerable group of adolescents would guide county policies and programs. While the data from this parallel survey cannot be combined with the phone-based sample due to different sampling techniques, our findings point to marked differences in this marginalized youth sample from the phone-based sample. This group has a greater proportion of racial minorities, has more youth reporting a history of physical emotional or mental limitations, and unstable housing. Sexual minority youth are also overrepresented. This is not surprising given known gay-related victimization and social barriers faced by sexual minority youth, which increases their likelihood of being unstably housed and marginalized [18].

These findings highlight several characteristics of RDD and convenience sampling to consider. Even surveys using probability-based recruitment methods are prone to under-represent more marginalized communities [19,20]. Data from our RDD survey are mostly reflective of the Allegheny County census of all ages (approximately 80% white, 13% black/African-American, 2% Hispanic Latino, and 5% other), though our sample likely recruited a slightly lower proportion of black/African-American youth (at 10%) [16]. Despite being an English-only survey, this underrepresentation did not extend to Hispanic/Latino youth. The mismatch, therefore, may reflect African-American-specific barriers to research participation in general [21] and the fact that we oversampled landline numbers when a greater proportion of African-Americans are likely to live in cellphone-only households [22]. It may be difficult to assess the needs of minority participants, even when properly sampled, in counties with small minority populations. Without oversampling minority groups, subgroup analyses of large data may not be informative [23]. Convenience sample precludes us from combining the two samples, and the marginalized youth sample size reflected resource limitations not ideal statistical subgroups. However, future youth surveys, whether county wide or school based should consider additional resources to oversample minority and marginalized populations if using phone-based methods. This may be important to understanding local health disparities. Local youth-serving agencies may use these data to focus their resources on the most affected youth and to understand within-minority group differences. Stakeholder engagement using participatory evaluation may facilitate the use of results by these decision makers [17]. Therefore, communities that do conduct school-based adolescent health behavior surveys should consider adding a marginalized youth sample to include youth often missed using school-based or RDD approaches.

There are limitations to this county-level adolescent health assessment. Convenience sampling employed for the marginalized youth population limits generalizability, and given different sampling techniques, the two data sets cannot be combined. However, the addition of this parallel survey draws attention to the health behaviors and needs of marginalized populations. The survey was self-report and thus limited by youth recall and social desirability bias. Privacy methods were intended to reduce this bias [24,25]. Offices of Human Subjects Research Protection in the United States and abroad may not agree to the waiver of parental permission. We hope that this study helps to promote the use of a justice and privacy-based argument when seeking such a waiver for adolescent risk surveys [12,13,26,27].

It is possible that the phone-based sample may have included some marginalized youth. We attempted to mitigate this by using consistent research assistants and with specific participant instructions; however, this could not be guaranteed. Both surveys were English only due to limited resources, a lack of validated items in other languages, and the small number of non-English-speaking youth in this county. Despite this, the expected proportion of Hispanic/Latino identifying youth completed the surveys. Our study lacks other markers of socioeconomic status, and those attempting to replicate these techniques may consider adding other variables.

The most recent YRBS school-based surveys showed 64%–90% student response rate and 99% overall quality control which includes completion rate [28]. Compared with our phone-based methods, school-based surveys have significantly better response and completion rates. With landlines and cell phone sampling frames, combined response rates were 38.4% and cooperation rates were about 50%. These rates were on par with the most recent National Survey of Children’s Exposure to Violence (NatSCEV II) by Finkelhor et al. [29] and accounting for recent survey trends [30]. Low response rates indicate possible biases, though we were unable to evaluate for these in the current study. However, our sample characteristics are similar to the target population demographics, and we weighted the sample to adjust for nonresponse. Therefore, we believe that bias resulting from a low response rate has been minimized. Survey data are countywide, so it is impossible to ascertain specific school-based or other granular geographic analysis. Schools interested in examining their student behaviors will need to conduct school-based surveys, and while still early, several of our local school districts have shown interest in doing so.

Finally, these methods cost approximately $200,000. In Massachusetts, however, the cost of surveying one large high school (2,500 students) was approximately $10,000 (K. Hacker, personal communication 2016). A sampling of half the 42 districts in our county would have a similar cost to our county-wide methods. Garnering each school board’s support would involve significant resources. Ideally, school districts would conduct the YRBSS regularly providing the best school-specific data, with higher response rates, and the ability to compile data at the county level. Alternatively, a random sampling of school classrooms—similar to statewide YRBSS methods but with larger samples—could be conducted which would likely be more cost-effective. We believe that having county-wide data will allow reluctant school systems to see the benefits of this information and thus invest in school-based data collection.

Despite these limitations, a countywide adolescent health behavior survey can be completed without school district participation. A broad representative sample can be collected and supplemented by sampling youth who would be missed on a phone survey, providing the county with data from a substantial proportion of youth. Multiple strategies ensured privacy and confidentiality and minimized coercion. The approaches described here provide reliable cross-sectional data in settings where there is resistance to school-based youth surveillance.

IMPLICATIONS AND CONTRIBUTION.

Representative adolescent health behavior data can be collected outside of school systems. This paper describes a unique combination of validated methods to gather these data among adolescents in one county. With these local data, public health departments and youth-serving agencies can direct limited resources to address local needs.