Development and cognitive testing of occupational safety and health questions for a youth survey: Addressing the research needs for a vulnerable working population

Abstract

Background:

While more than 2.5 million U.S. high students worked in 2020, data to assess how work affects this group are sparse. To facilitate such research, a set of occupational safety and health questions for inclusion on the Youth Risk Behavior Surveillance System (YRBSS) and other youth-focused surveys was developed.

Methods:

Survey questions about occupational experiences of young workers were adapted from other surveys or created de novo. Key audiences were engaged to define priority topic areas and develop draft questions, which were further refined through cognitive interviews with working youth.

Results:

Twenty-one resulting questions spanned multiple work-related topics: employment status; health outcomes; psychosocial exposures; and safety climate. Cognitive testing revealed that youth (aged 14–19) had difficulty with temporal concepts. Some difficulties reflected the propensity of youth to engage in multiple, online, and informal jobs. During three rounds of interviews, questions were adjusted to better reflect youth employment circumstances and language. Four states added at least one work-related question to their 2023 Youth Risk Behavior Survey questionnaire, and the full set of questions has been disseminated to federal agencies and partners.

Conclusions:

Including tailored questions about employment in surveys of youth will facilitate occupational health surveillance for this group. Analysis of resulting data can help to close knowledge gaps, provide current prevalence data, inform policy, and allow development of focused prevention and intervention strategies to reduce adverse outcomes among young workers.

Young workers are a vulnerable segment of the workforce. Approximately 17% of U.S. high school students aged 16–19 years worked while enrolled at school in 2020.¹ While employment helps adolescents develop job skills, build character, and work towards financial independence, young workers are at risk for workplace illnesses, injury, violence, and bullying.^2–4 Nearly every minute, a U.S. worker aged 15–24 years is injured,⁵ and one-third of a sample of approximately 1,200 U.S. high school students reported experiencing workplace violence (verbal threats, physical attacks, and sexual harassment).⁴ Workplace violence can have a negative effect on the mental health of young workers and lead to feelings of poor self-worth, anxiety, depression, and use of maladaptive coping strategies like smoking.⁶

According to Bureau of Labor Statistics (BLS) data, approximately 400 workers aged 19 years and younger died as a result of occupational injuries during 2016–2020.⁷ Workers in this age group also had an estimated 81,500 nonfatal occupational injuries and illnesses treated in U.S. hospital emergency departments in 2020.⁸ Like adults, young workers encounter a range of occupational hazards through their varied employment settings, with observed health effects including decreased hearing acuity following exposure to high levels of occupational noise and certain chemicals;⁹ health-related illness symptoms among Latinx child farmworkers;¹⁰ and illnesses (e.g., asthma), poisonings, and injuries from common substances ranging from pool chemicals to cleaning agents,^{11, 12} but may have less access to safety training and resources compared to adult workers.¹³

While data sources like the BLS Survey of Occupational Injuries and Illnesses (SOII) and the work supplement to the National Electronic Injury Surveillance System (NEISS) capture injuries to young workers, they are known to substantially under ascertain the total injury burden.^14–16 Young workers have been found to underestimate the dangers associated with work tasks, thus increasing their risk for injury.¹⁷ Moreover, information about other occupational health concerns such as bullying and verbal harassment are beyond the scope of these instruments; therefore, compiling a more complete picture of the aggregate risks of youth work requires piecing together information from multiple sources, a process hindered by issues such as differences between sample populations and case ascertainment methods.

The current project was designed to address research gaps related to youth employment by developing cognitively tested questions on multiple topic areas for inclusion in Centers for Disease Control and Prevention’s (CDC’s) Youth Risk Behavior Surveillance System (YRBSS) and other youth-focused surveys.

METHODS

Surveys of youth are administered to different populations and have different formats. The ultimate goal of this study was to develop cognitively tested work-related questions for inclusion in a range of studies. However, a specific questionnaire format was needed in order to create the questions, so selection of a primary target survey was essential. YRBSS was selected because it: 1) comprises a nationally representative survey and local surveys; 2) is a cross-sectional survey administered every two years, with frequent potential to add new questions; 3) has no financial cost to propose the addition of new questions; 4) is similar to other CDC surveys that NIOSH has worked with extensively, including adding work-related questions (e.g., Behavioral Risk Factor Surveillance System, National Health Interview Survey); and 5) features concise, stand-alone question format easily adaptable for other surveys.

Youth Risk Behavior Surveillance System (YRBSS)

YRBSS is a system of surveys. The main aim of YRBSS is to monitor health-related behaviors that contribute to the leading causes of death, disability, and social problems among U.S. youth. This aim is achieved through administration of biennial surveys known as Youth Risk Behavior Surveys (YRBS) to U.S. high school students. These biennial surveys include a national survey conducted by CDC and state, territorial, and tribal agencies; and local surveys conducted by state, territorial, tribal, and local education and health agencies.¹⁸ The YRBSS was established in 1990 and collects a range of information, including youth demographics, health-related behaviors, health outcomes, and school performance, but has never included questions about employment experiences or work-related health outcomes

Certain survey design features are particular to YRBSS.¹⁹ YRBSS allows participating states, territories, tribes, and school districts (collectively referred to as sites) to modify a standard questionnaire within certain parameters to create their local surveys.¹⁸ Sites that want to add questions can select from a list of optional questions. Because sites may not administer all questions on a topic when they are building their survey from the optional questions list, YRBSS questions must be able to stand alone. The survey is self-administered by high school and middle school students at school. Therefore, brief introductions are sometimes necessary to explain the purpose of the question and provide pertinent additional information for the respondent to consider when providing an answer. YRBSS questions are restricted to multiple choice format, with a maximum of eight close-ended response choices (i.e., no free text responses permitted). In addition, no routing through the survey is permitted; every student must answer every question, so each question had to include an option for students to indicate they had not worked during the time period of interest.

Question Development

The process of developing and cognitively testing a set of survey questions addressing occupational safety and health-related issues faced by young workers (aged 14–19 years) had several components (Figure 1). Draft questions were developed based on the limited literature on occupational safety and health of youth, scans of existing work-related surveys, and input from key audiences. The draft questions then underwent a question design review stage and pretesting of the survey questions using cognitive interviews.

Figure 1.

Overview of the process to create and cognitively test the occupational safety and health questions for youth.

Throughout the question development process, attention was paid to the literature on designing and administering survey questions to youth, along with the specific design requirements of YRBSS. A key finding of the literature is that adolescents may not understand, interpret, or react to the world around them in the same way as adults do.²⁰ Survey questions therefore need to connect with the young person’s reality, and concepts should be tangible and concrete.²¹ Furthermore, certain traits, while present also among adult survey takers, are often more prominent among youth.²² For example, young people may be more likely to satisfice (apply the minimum cognitive effort to produce an answer);²² be influenced by context;²³ use vague quantifiers;²⁴ and struggle with reference periods²⁵ or questions that contain more than one topic.²¹ In addition, literacy levels vary, and adolescents will often try to respond to survey questions, regardless of whether they know the answer.²⁰

Identification of question topic areas had two components: a literature review and input from key audiences. The literature review identified broad topic areas with gaps in knowledge about occupational safety and health among young workers. To ensure that topics selected reflected critical, current concerns and that the topic areas and questions would reflect a range of key audience research priorities, YRBS coordinators, National Institute for Occupational Safety and Health (NIOSH) and academic researchers, and state partners with expertise in the areas of sleep and fatigue, work-related injuries, youth agricultural work, workplace violence, and occupational exposures were recruited to participate in the question development process. Ultimately, with key audience input, employment status, health outcomes, job type, psychosocial exposures, and safety climate were selected as the topic areas. Key audience workgroups were formed to participate in question development for each of these areas.

Development of draft questions in these areas comprised several steps. Existing surveys with work-related questions were evaluated to determine whether any questions could be used as is or serve as templates. Surveys focused on adults and/or youth were considered, with the understanding that any questions designed for adult respondents might need restructuring for a youth audience. Several questions related to occupational safety and health topic areas of interest were extracted from the surveys for further consideration (Appendices A and B). These questions, in some cases modified for the youth audience, along with de novo questions developed by the key audience workgroups, formed the pool of draft questions.

To obtain feedback from the key audience workgroups on draft questions, a method similar to the Delphi technique was used. The Delphi technique is a group communication process that relies on multiple rounds of iterative feedback via a structured questionnaire.^{26, 27} It has been employed within different disciplines, including health sciences, as a means of reaching consensus by providing all participants the opportunity to share their suggestions. For each question topic area, the questions were sent to the workgroup, along with an introduction and a response form to gather specific feedback. In some cases, a question had multiple versions and key audiences were polled to determine which version they felt best captured the construct. Additionally, questions related to response options were asked, such as how specific cut-points should be created (e.g., for hourly work) or whether the response options were appropriate for high school students. Each topic area went through two rounds of feedback before the questions were finalized for cognitive testing. Ultimately, twenty-six questions were developed (six employment status; four health outcome; six job type; three psychosocial exposure; and seven safety climate). For the cognitive testing process, a brief description of each question was provided to explain the intent behind each question.

Cognitive Testing

The proposed questions were evaluated by the National Center for Health Statistics’ (NCHS) Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER), in collaboration with NIOSH. Consent procedures and the cognitive interviewing protocol were approved by the NCHS Ethics Review Board, CDC’s Human Subjects review, and Office of Management and Budget. An initial review of the draft questions was carried out by CCQDER staff experienced in survey question design. The survey questions were then pretested using one-on-one cognitive interviewing methodology, considered a staple of survey questionnaire development.²⁸ The questions were verbally administered to youth by the cognitive interviewer and cognitive interviewing techniques applied to explore construct validity, thus providing an understanding of question performance and informing the question design.

Thirty-five cognitive interviews were conducted in total during the autumn of 2021 with English-speaking high school students who had worked at a paying job within the past 12 months. Cognitive interview respondents were purposively sampled and screened, in an attempt to reflect different demographics found within the population of interest (age, school grade, sex, race and ethnicity) and characteristics relevant to the topics covered by the survey questions (working status and type of work). Table 1 displays the respondent demographics which appear skewed towards students who were non-Hispanic White, female, living in non-rural areas, and in school grades 11 and 12. However, the distribution of demographic characteristics is similar to that of the national 15–19-year-old working population with the exception of females.²⁹

Table 1.

Demographics of cognitive interview respondents.^*

Demographic grouping	Total number of respondents (N=35)	Percentage (%) of total respondents**
Age group (years)
14–15	9	25.7
16–18	26	74.3
School grade
9–10	9	25.7
11	12	34.3
12	14	40.0
Gender
Female	24	68.6
Male	11	31.4
Race/Ethnicity
Hispanic	6	17.1
Non-Hispanic	29	82.8
Asian or Multiple Race Groups	6	17.1
Black or African American	6	17.1
White	17	48.6
Rurality
Rural	5	14.3
Not Rural	30	85.7

^*Table adopted from the full NCHS report by Amanda Wilmot entitled, “Cognitive Interview Evaluation of Questions on Occupational Safety and Health for the Youth Risk Behavior Surveillance System (YRBSS).” Available at https://wwwn.cdc.gov/qbank/report.aspx?1228.

^**Percentages may not total to 100% due to rounding.

COVID-19 closures and restrictions necessitated adoption of remote video-platform interviewing. This format permitted greater geographical diversity and greater access to the cognitive interviews for this group of busy high school students compared to a more traditional in-person cognitive interviewing mode.³⁰ Verbal consent to take part in the study was obtained from youth aged 18 or over (all interviewees resided in states where the age of majority was 18). For minors, written consent was first obtained from a parent or guardian, followed by verbal assent from the youth. Each respondent received $40 cash for the interview, which lasted at most one hour. A few of the questions dealt with sexual assault or harassment in the workplace, as well as other types of harassment (physical, verbal, and bullying); in addition to being told at the start of the interview that that they could skip any question they chose or stop the interview at any time, prior to the administration of these potentially sensitive questions respondents were also told that they did not need to provide a response because it was their understanding of the question that was of primary interest.

Throughout, standards and guidance for conducting cognitive interviewing studies recommended by the Office of Management and Budget³¹ and outlined by Miller (2017),³² were adhered to, incorporating the various checks and balances required throughout the life cycle of a cognitive interviewing study. This included taking an iterative approach to the evaluation, considered best practice.^{31, 33, 34} The questions were tested, revised and retested across three rounds of cognitive interviewing, each round comprising separate samples of respondents, before final amendments were made based on the study findings. Further details about how the cognitive interview methodology was applied, along with a description of the consent process and other requirements of the NCHS Ethics Review Board, can be found in the full cognitive interview study report.³⁵

RESULTS

In total, twenty-six draft questions were developed for cognitive testing. Ten were modelled on items identified during our scan of other surveys including work-related data (Appendices A and B). Several questions were adapted from other youth surveys: basic questions about employment status, hours worked, and job type from Washington State 2010 Healthy Youth Survey, Massachusetts 2009 Youth Health Survey, and New Hampshire 2017 YRBS; and an item about unpaid or volunteer work from the Canadian Health Survey on Children and Youth (formal volunteering section) and a 2011 Connecticut School Health Survey question. Questions about work-related injury and safety climate combined input from youth (the Washington State, Massachusetts, New Hampshire instruments) and adult (National Health Interview Survey) surveys. Additional questions about shift work and psychosocial factors were adapted from the 2015 National Health Interview Survey Occupational Health Supplement, while a question on work stress was adapted from the Quality of Worklife (QWL) questionnaire. Two questions about fatigue during work or caused by work were adapted from the Patient Health Questionnaire-9 (PHQ-9) modified for teens. Finally, a question designed to capture migrant farm work performed by the student or the student’s family members used the U.S. Department of Labor’s definition of migrant work. Other questions were developed de novo to cover topics for which we could not find pertinent questions from existing surveys or thought additional information was needed.

As the development process progressed five questions were dropped from the testing based on relevance and design for youth. For example, a question on contingent work was eliminated following initial review because age eligibility restrictions for jobs involving transporting passengers meant that few in the sample population of high school students would be eligible. Other questions were eliminated following Round 1 testing because they seemed repetitive to respondents or utilized fewer concrete concepts and vague quantifiers, yielding multiple interpretations.

Cognitive testing revealed both overall tendencies in responses to questions and issues related to specific draft questions. Both are detailed in the full report on the cognitive interviewing stage of the process;³⁵ here, we review some overall tendencies. A primary issue was the definition of a paid job. The questions were designed to include work in a family business (restaurant, farm), even if the respondent was not compensated directly, but excluded work within the family (i.e., babysitting, lawncare), even if compensated with an allowance. Students generally understood this distinction, but sometimes discounted jobs for pay that occurred infrequently/irregularly. Respondents’ eagerness to provide an answer sometimes led them to give responses that reflected the job they felt most salient to the question topic, regardless of whether that job corresponded to the intent or time scope specified in the question. Respondents frequently minimized the cognitive burden associated with tasks, especially those involving computation (e.g., total hours worked, particularly for respondents working multiple jobs and/or shifts).

Load minimizing tactics such as reporting daily, rather than weekly hours, as well as estimation, tended to lead to underestimates of hours worked. Given the fluctuations in number of jobs and hours (erratic and seasonally variable schedules) worked by youth, particularly during the school year, providing a limited set of categorical responses (range of hours) and emphasizing the time period referenced by including it in response options, as well as the stem, are helpful but may still yield responses that do not accurately reflect the hours worked. Respondents sometimes confused prompts asking for information “in the past month,” thinking about the last calendar month, rather than the past 30 days. Substituting “during the past 30 days” improved the accuracy of their responses but did not completely correct misunderstandings about the timeframe. Some types of work (e.g., working within the respondent’s family, babysitting for other families, working directly for clients, making crafts to sell online) led to confusion when respondents were given questions involving supervisors or managers.

Some concepts proved more difficult for youth to grasp, as their life experiences were not aligned with those of adults. For example, respondents’ concept of time of “day” sometimes varied from that of adults when considering their work shift-pattern, since they were not at work but in school during the daytime. Certain terms that are commonplace for adults were opaque to some of the respondents, necessitating revision during the cognitive testing process. The description of “agricultural work,” for example, was changed to “work you have done on a farm or ranch” because a few respondents were unable to assign meaning to agricultural work. Finally, respondents were often uncomfortable with ambiguity; use of examples and detailed descriptions in response options seemed to mitigate this problem.

The third round of cognitive testing yielded 21 finalized questions. The full list of questions is available for all interested researchers (Appendix C).

DISCUSSION

No existing survey captures comprehensive information about employment and health in young workers, particularly those still in high school. The resulting research gap hinders focused protection of this workforce. Data from an augmented YRBS, and from addition of questions about work to other surveys of youth, could fill many gaps in understanding the associations between multiple aspects of youth employment and health-related behaviors and outcomes.

The national YRBS is representative of the national population of 9^th through 12^th grade students in public and private schools and obtains data on nearly 15,000 students during each biennial survey cycle.¹⁸ In the most recent survey cycles, the national YRBS included questions on average hours of sleep per night; academic performance in school; suicide and suicidal thoughts; tobacco product use (cigarettes and electronic vapor products); alcohol use; drug use (marijuana, cocaine, opioids); and other topics. Adding questions on work status, type, schedules, and experience, as well as work-related health and psychosocial outcomes, to this representative survey of U.S. teens with varied demographic backgrounds would facilitate evaluation of behavioral differences between working and non-working high school students and the effects of work and work hours on students’ behavioral health. In addition, work-related questions could be used to assess questions about health equity. For example, comparison of prevalences of fatigue, school performance, and having experienced workplace violence between students who work to help support their families versus those working for other reasons could enhance understanding of the experience of students whose families may be financially stressed. In the area of workplace safety, addition of work-related questions about injuries to YRBS and other surveys administered to large numbers of students could yield population prevalence estimates to supplement sources such as SOII and NEISS-Work.

Young workers differ from adult workers, and some of those differences were reflected in the cognitive testing process for this set of questions for young workers. In some ways, the experience of young workers more closely reflects newer modes of work which are increasingly prevalent among adults, with greater participation in online work, gig, and informal work and holding multiple part-time jobs simultaneously. Young workers are more likely to be seasonally employed, and to change jobs with the seasons (specifically school year vs. summer), than many adult workers.³⁶ These characteristics of youth work led to greater difficulties with questions involving defining a primary job, or determining which obligations comprised paid work, along with certain traits associated with the survey question response process that are more pronounced when surveying youth. Specifying a timeframe in questions related to youth employment is therefore both important and complex; types of jobs, reasons for working, and work hours, arrangements, and exposures all likely vary with greater frequency than among many adult workers.³⁷ Determining whether to elicit responses reflecting the primary job or all paid work is also key. Students are more likely than adult workers to be engaged in temporary and contract work.³⁷ In addition, many students undertake volunteer work, so a question about the extent of voluntary work was developed as part of this project. Whether and how to include these types of work in surveys of youth must be considered.

The literature on how youth interact with the survey question and answer process is relatively sparse but is generally consistent;³⁸ the findings should be taken into consideration during the design and development phase, along with pretesting of the survey questions (ideally using cognitive testing methods), since questions that are effective for use with adults may not work in the same way with youth.

The YRBS program allows submission of a maximum of three questions for potential placement on the national survey from each entity proposing new questions. Determination of which of the pool of all proposed new questions will be added to the national survey occurs through a voting process, with majority approval required for addition. Proposed questions not meeting this bar are then presented to sites for optional addition to their YRBS questionnaires.

From the list of 21 new questions, the three submitted to the YRBS program were selected because they were considered applicable to most working teens and therefore would be most likely to yield statically reportable results separately or if used in conjunction with other YRBS questions (items concerning sleep, mental health, etc.). The three questions focus on employment status, reason for employment, and the impact of employment on daily life and provide necessary, basic background data on youth and employment. Although the questions were not adopted for the 2023 national YRBS questionnaire, four states added at least one of the three questions to their 2023 YRBS questionnaires, and two states reported that they would be likely to include them in future YRBS survey cycles. Multiple sites had not yet finalized their 2023 YRBS questionnaire at the time this manuscript was written, so the number of sites actually administering these questions may be larger, and the questions will be resubmitted for consideration for future YRBS survey cycles. The full set of 21 questions has since been shared with federal agencies and other partners that administer health-related questionnaires to youth.

The 21 questions developed from this effort cover a wide range of topics related to youth employment and performed well by the end of the cognitive testing process. Although we intentionally developed questions using a stand-alone format, researchers considering including the questions on surveys with more flexible formatting might consider some format modifications to better address the complex world of youth work. Where skip patterns are permitted, the “did not work” response that was necessary for the YRBS format could be dropped from questions that should be bypassed for students who have not worked in the specified timeframe. If questions do not need to stand alone, question stems could be simplified, and section introductions expanded to include more examples and clarification. Finally, although the types of work youth are engaged in is more limited than the universe of adult jobs, fully free text responses for industry and occupation, recommended for surveys of adults, could be useful for youth surveys as well.

Limitations

The cognitive interview findings may not reflect the full range of experiences of the target population of national YRBS or other surveys of youth. For example, despite efforts to recruit youth who might have performed migrant work through contacts with organizations serving migrant workers, no respondents said they had done so; this item and others intended to address the experience of specific groups of students might require further refinement. Although the use of remote interviewing using video conferencing software made it easier for busy youth to take part in this study, it is worth noting that some young people who have limited or no internet access or who do not have voice and video capabilities are excluded from studies carried out this way. All interview participants spoke English. Additional testing would be required for inclusion of these questions in instruments written in other languages to ensure cross-cultural equivalence. YRBS is self-administered. However, cognitive testing focused on an evaluation of the question-and-answer process from the perspective of youth; the YRBS form filling task was not tested as part of this study and would be better suited to separate investigation with a full, near-final version of the survey questionnaire. Finally, students complete the YRBS at school; surveys that include youth in this age group who are no longer in school may need to retest and adjust some questions.

CONCLUSIONS

The questions developed through this project address the need for data on the work status, work schedules, occupational health risks, behaviors, and outcomes among the high-risk, vulnerable population of young workers. Development and inclusion of occupational safety and health questions on youth surveys, and analysis of resulting data, can help to close knowledge gaps, provide current prevalence data, inform policy, and allow for better prioritization of methods to reduce adverse outcomes among young workers.

IMPLICATIONS FOR SCHOOL HEALTH POLICY, PRACTICE, AND EQUITY.

The cognitively tested questions resulting from this project can be used to assess a wide range of occupational safety and health issues among young workers. Young workers are understudied and vulnerable, so more data are crucial to understand the impact of work on their health and well-being.

APPENDICES

Appendix A. Description of existing youth or school-based surveys and their corresponding survey questions used in the development of occupational safety and health questions for youth.

New Hampshire Youth Risk Behavior Survey (YRBS)

The New Hampshire YRBS provides health-related information about New Hampshire high school students (grades 9–12) and is jointly administered by the Departments of Health and Human Services and Education.

Weblink: https://www.dhhs.nh.gov/programs-services/population-health/health-statistics-informatics/youth-risk-behavior-survey

Question adopted from 2017 New Hampshire YRBS:

During the past 12 months, which one of the following best describes the place where you most recently worked for pay? (Do not count chores, babysitting, or yard work such as raking leaves, shoveling snow, or mowing grass.)

1. I did not work for pay during the past 12 months

2. Restaurant, including a fast-food restaurant, pizza place, coffee shop, or ice cream shop

3. Grocery store or supermarket or other type of store, such as a clothing store, gas station, pharmacy, or pet store

4. Nursing home, hospital, clinic, or doctor’s office

5. Golf course, camp, sports center, swimming pool, amusement park, or movie theater, or other place people go for fun

6. Construction site

7. Farm

8. Some other place

Canadian Health Survey on Children and Youth

The Canadian Health Survey on Children and Youth provides information on the health and well-being of Canadian children and youth (ages 1–17 years). The questionnaire was developed by Statistics Canada in collaboration with the Public Health Agency of Canada and Health Canada.

Weblink:https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&Id=282165

Question adopted from 2016 Canadian Health Survey on Children and Youth:

In the past 12 months, did you do unpaid volunteer activities for any organization?

1. Yes

2. No

Connecticut School Health Survey

The Connecticut School Health Survey provides health-related information on Connecticut high school students (grades 9–12) and is administered by the Department of Public Health.

Weblink: https://portal.ct.gov/DPH/Health-Information-Systems--Reporting/Hisrhome/Connecticut-School-Health-Survey#:~:text=The%20Connecticut%20School%20Health%20Survey%20%28CSHS%29%20is%20a,known%20as%20the%20Youth%20Risk%20Behavior%20Survey%20%28YRBS%29.

Questions adopted from 2011 Connecticut School Health Survey:

During an average week when you are in school, how many hours do you work at a paying job outside your home?

1. 0 hours

2. 1 to 4 hours

3. 5 to 8 hours

4. 9 to 12 hours

5. 13 to 20 hours

6. 21 or more hours

During the past 30 days, how many hours did you spend helping other people without getting paid (such as helping out at a hospital, daycare center, food pantry, youth program, or community service agency or doing other things) to make your community a better place for people to live?

1. 0 hours

2. 1 to 4 hours

3. 5 to 8 hours

4. 9 to 12 hours

5. 13 to 20 hours

6. 21 or more hours

Washington State Healthy Youth Survey

The Washington State Healthy Youth Survey provides health-related information about youth (grades 6 – 12) in Washington State and is led by the Office of the Superintendent of Public Instruction, the Department of Health, the Health Care Authority’s Division of Behavioral Health and Recovery, and the Liquor and Cannabis Board.

Weblink: https://doh.wa.gov/data-statistical-reports/data-systems/healthy-youth-survey

Questions adopted from 2010 Washington Healthy Youth Survey:

How many hours per week are you currently working for pay, NOT counting chores around your home, yard work, or babysitting?

1. None – not currently working

2. 10 hours a week or less

3. 11–30 hours per week

4. 31–40 hours per week

5. More than 40 hours per week

NOT counting chores around your home, while working for pay, have you ever been injured badly enough that you needed to go to a nurse, doctor, or hospital?

1. Never worked for pay

2. Have worked, but never been injured enough to see a nurse or doctor

3. Yes, I was injured at work and needed to go to a nurse, doctor, or hospital

How would you describe the place that you currently work? (Pick your main job. Choose one.)

1. Not currently working

2. Restaurant (including fast food)

3. Store (including grocery, convenience, clothing, music, or gift stores, gas station)

4. Hospital, clinic, or nursing home

5. Construction

6. Farm or dairy

7. Factory

8. Packing house or food processing

9. Babysitting

10. Yard work

11. Other

Patient Health Questionnaire – 9 (PHQ-9)

The PHQ-9 is a multi-purpose, validated tool used in primary care environments to screen for the presence and severity of depression. The Health Resources and Services Administration created a modified version of PHQ-9 for teenagers in 2010.

Weblink: https://www.hrsa.gov/behavioral-health/phq-9-modified-teens

Questions adopted from PHQ-9 (version 2, 2010):

How often have you had trouble falling asleep, staying asleep, or sleeping too much during the past two weeks?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

How often have you been feeling tired, or having little energy during the past two weeks?

1. Not at all

2. Several days

3. More than half the days

4. Nearly every day

Massachusetts Youth Health Survey

The Massachusetts Youth Health Survey provides health-related information about youth (grades 6–12) in Massachusetts and is conducted by the Department of Health in collaboration with the Department of Elementary and Secondary Education.

Weblink: https://www.mass.gov/lists/massachusetts-youth-health-survey-myhs

Questions adopted from 2009 Massachusetts Youth Health Survey:

In the past 12 months, have you ever worked at a job for pay for someone other than your parent or guardian, NOT including babysitting or yard work?

1. Yes

2. No (skip pattern)

In the past 12 months, while you were working for pay, were you ever injured on the job badly enough that you needed to go to a nurse, doctor, or hospital?

1. Yes

2. No

Which best describes the place you most recently worked at? (If you work in more than one place, choose the place you work the most hours.)

1. Restaurant (such as fast food, pizza places, coffee shops, ice cream shops)

2. Grocery store or Supermarket

3. Other retail store (places where things are sold such as clothing stores, gas stations, pharmacies, pet stores)

4. Health care facility (such as nursing homes, hospitals, clinics, doctors’ offices)

5. Recreation or entertainment place (such as golf courses, camps, sports, amusement parks, movie theaters)

6. Construction site

7. Landscaping company

8. Other (describe that place:______)

Appendix B. Description of existing adult surveys and their corresponding survey questions used in the development of occupational safety and health questions for youth.

National Health Interview Survey – Occupational Health Supplement (NHIS-OHS)

In 2015, NIOSH sponsored an occupational health supplement in the National Health Interview Survey to gather information on workplace exposures and work-related health conditions among U.S. adult workers.

Weblink: https://www.cdc.gov/niosh/topics/nhis/default.html

Questions adopted from the 2015 NHIS-OHS:

Which of the following best describes the hours you usually work?

1. A regular daytime schedule

2. A regular evening shift

3. A regular night shift

4. A rotating shift

5. Refused

6. Don’t know

The demands of my job interfere with my personal or family life.

1. Strongly agree

2. Agree

3. Disagree

4. Strongly disagree

5. Refused

6. Don’t know

Overall, how safe do you think your workplace is? Would you say…

1. Very safe

2. Safe

3. Unsafe

4. Very unsafe

5. Refused

6. Don’t know

The health and safety of workers is a high priority with management where I work.

1. Strongly agree

2. Agree

3. Disagree

4. Strongly disagree

5. Refused

6. Don’t know

During the past 12 months, were you threatened, bullied, or harassed by anyone while you were on the job?

1. Yes

2. No

3. Refused

4. Don’t know

The General Social Survey (GSS) – Quality of Worklife (QWL) Survey

The General Social Survey is a nationally representative survey of U.S. adults that collects information on American society to examine trends in opinions, attitudes, and behaviors. It is conducted by The National Data Program for Social Sciences at NORC University of Chicago. The Quality of Worklife survey module is asked of all GSS respondents who work in a given survey year.

Weblink: https://gss.norc.org/Pages/quality-of-worklife.aspx

Question adopted from GSS-QWL:

Do you find your work stressful?

1. I do not have a paying job.

2. Always

3. Often

4. Sometimes

5. Hardly ever

6. Never

Appendix C. Twenty-one cognitively tested, finalized occupational safety and health questions for youth surveys.

INTRO1: The next set of questions ask about your paying jobs, including work at a family business or farm, even if you were not paid directly. When answering these questions do NOT include chores, yard work or babysitting at your own home.

• 1
  What is the main reason you worked at a paying job during the past 12 months? (Select only one response)*

  1. I have not worked at a paying job during the past 12 months
  2. To help my family pay bills or buy food
  3. To save money for my future after I graduate high school
  4. To earn day-to-day spending money for myself
  5. To learn new skills
  6. Some other reason
• 2
  Last summer during school break, how many hours per week did you usually work at all your paying jobs?

  1. I did not work at a paying job last summer
  2. 1 to 5 hours per week
  3. 6 to 10 hours per week
  4. 11 to 20 hours per week
  5. 21 to 30 hours per week
  6. 31 or more hours per week
• 3
  During the school year, how many hours per week do you usually work at all your paying jobs?*

  1. I do not work at a paying job during the school year
  2. 1 to 5 hours per week
  3. 6 to 10 hours per week
  4. 11 to 20 hours per week
  5. 21 to 30 hours per week
  6. 31 or more hours per week
• 4
  During the school year, how often, if at all, do you have less time to spend on schoolwork, family responsibilities (such as chores), or extracurricular activities (such as sports or clubs) because of working at a paying job?*

  1. I do not work at a paying job during the school year
  2. Never
  3. Sometimes
  4. Most of the time
  5. Always

INTRO2: Think about your paying jobs during the school year. If you have more than one paying job think about the job you work the most hours.

• 5
  During the school year, when do you usually work at your paying job during the school week (Monday to Friday)? If you have more than one paying job during the school week think about the job you work the most hours.

  1. I do not work at a paying job during the school week
  2. Between 7am and 4pm (Daytime)
  3. Between 4pm and 11pm (Evenings)
  4. Between 11pm and 7am (Nights)
  5. When I work during the school week changes (for example, from daytime to evenings)
• 6
  During the school year, when do you usually work at your paying job on weekends (Saturday and Sunday)? If you have more than one paying job on weekends think about the job you work the most hours.

  1. I do not work at a paying job on weekends
  2. Between 7am and 4pm (Daytime)
  3. Between 4pm and 11pm (Evenings)
  4. Between 11pm and 7am (Nights)
  5. When I work on weekends changes (for example, from daytime to evenings)
• 7
  During the school year, which one of the following best describes the place where you work at your paying job? If you have more than one paying job during the school year think about the job you work the most hours.

  1. I do not work at a paying job during the school year
  2. Restaurant or fast-food place
  3. Retail store, grocery store or pharmacy
  4. Recreation or entertainment place
  5. Farm or ranch (including family-owned)
  6. Office work
  7. Babysitting, landscaping or yard work done outside of your own home
  8. Some other place
• 8
  While working at your paying job during the school year, have you received any on-the-job training about how to protect yourself from workplace bullying, violence, or sexual harassment?

  1. I do not work at a paying job during the school year
  2. Yes
  3. No
• 9
  While working at your paying job during the school year, do you feel that you could talk to your supervisor or manager about any workplace bullying, violence, or sexual harassment?

  1. I do not work at a paying job during the school year
  2. I work at a paying job but do not have a supervisor or manager
  3. Yes
  4. No
• 10
  While working at your paying job during the school year, have you received any safety training about how to protect yourself from being injured at work?

  1. I do not work at a paying job during the school year
  2. Yes
  3. No
• 11
  While working at your paying job during the school year, do you feel you could talk to your supervisor or manager about an unsafe work task you might be asked to do?

  1. I do not work at a paying job during the school year
  2. I work at a paying job but do not have a supervisor or manager
  3. Yes
  4. No
• 12
  How often, if at all, do you find working at your paying job to be stressful?

  1. I do not work at a paying job
  2. Never
  3. Sometimes
  4. Most of the time
  5. Always

INTRO3: Think about all of your paying jobs during the past 12 months

• 13
  During the past 12 months, while working at a paying job, have you been injured badly enough on the job that you needed to go to a nurse, doctor, or hospital?

  1. I did not work at a paying job during the past 12 months
  2. Yes
  3. No
• 14
  During the past 12 months, have you taken prescription pain medication (such as Vicodin, Oxycontin or Percocet) for an injury that happened while you were working at a paying job?

  1. I did not work at a paying job during the past 12 months
  2. I did not have a workplace injury during the past 12 months
  3. Yes, I took prescription pain medication for my workplace injury
  4. No, I did not take prescription pain medication for my workplace injury

INTRO4: Think about all your paying jobs during the past 30 days

• 15
  During the past 30 days, how often, if at all, has working at a paying job, in addition to completing schoolwork, family responsibilities (such as chores), or extracurricular activities (such as sports or clubs), made you feel tired or worn out?

  1. I did not work at a paying job during the past 30 days
  2. Never
  3. Sometimes
  4. Most of the time
  5. Always

INTRO5: Think about all the paying jobs you have ever had outside of your own home.

• 16
  While working at a paying job, have you ever been sexually harassed or sexually assaulted, for example, been inappropriately touched, had sexual comments made about how you look, been shown photos or videos of a sexual nature?

  1. I have never had a paying job
  2. Yes
  3. No
• 17
  While working at a paying job, have you ever been physically assaulted, for example, been hit, pushed, kicked, or slapped?

  1. I have never had a paying job
  2. Yes
  3. No
• 18
  While working at a paying job, have you ever been verbally abused or bullied, for example, been name called or insulted, threatened with physical harm, screamed, or yelled at with offensive language, had rumors spread about you?

  1. I have never had a paying job
  2. Yes
  3. No

INTRO6: The next question asks about any unpaid volunteer or community service work you have done in the past 30 days.

• 19
  During the past 30 days, how many hours did you spend doing any unpaid volunteer or community service work?

  1. I did not do any unpaid volunteer or community service work during the past 30 days B. 1 to 5 hours
  2. 6 to 10 hours
  3. 11 to 20 hours
  4. 21 to 30 hours
  5. 31 or more hours

INTRO7: The next two questions ask about work you have done on a farm or ranch.

• 20
  During the past 12 months, have you worked on a farm or ranch? Do not include volunteer or community service work or club activities like 4H.

  1. No
  2. Yes, I worked on a farm or ranch that my family owns
  3. Yes, I worked on a farm or ranch that my family does not own
• 21
  During the past 3 years, have you moved to a different location for part of the year in order to work for pay on a farm or ranch?

  1. Yes
  2. No

*Question was submitted in December 2021 to the YRBSS program for consideration to be added to the 2023 national YRBS questionnaire.