Skip to main content
American Journal of Public Health logoLink to American Journal of Public Health
. 2002 Feb;92(2):235–237. doi: 10.2105/ajph.92.2.235

Classroom-Based Surveys of Adolescent Risk-Taking Behaviors: Reducing the Bias of Absenteeism

Sally Guttmacher 1, Beth C Weitzman 1, Farzana Kapadia 1, Sharon L Weinberg 1
PMCID: PMC1447049  PMID: 11818298

Abstract

Objectives. This investigation examined the effectiveness of intensive efforts to include frequently absent students in order to reduce bias in classroom-based studies.

Methods. Grade 10 students in 13 New York City high schools (n = 2049) completed selfadministered confidential surveys in 4 different phases: a 1-day classroom capture, a 1-day follow-up, and 2 separate 1-week follow-ups. Financial incentives were offered, along with opportunities for out-of-classroom participation.

Results. Findings showed that frequently absent students engaged in more risk behaviors than those who were rarely absent. Intensive efforts to locate and survey chronically absent students did not, however, significantly alter estimates of risk behavior. Weighting the data for individual absences marginally improved the estimates.

Conclusions. This study showed that intensive efforts to capture absent students in classroom-based investigations are not warranted by the small improvements produced in regard to risk behavior estimates.


At present, most research on adolescent risk behaviors is school based1,2 and involves either a 1- or 2-day classroom-based survey methodology, with investigators surveying students in the classroom at a specific time on a given day. In urban public high schools with high rates of daily absenteeism, these survey protocols have the potential to produce considerable bias in that large numbers of students are not included. Rates of classroom absenteeism (i.e., students skipping individual classes) tend to be even higher and more difficult to measure accurately. Furthermore, students who are chronically absent represent an additional challenge to obtaining representative data. Researchers have relied on a variety of techniques, including tracking respondents through friends, conducting intensive follow-up sampling, and weighting data according to self-reported absences, to address bias resulting from absenteeism (P. M. O'Malley, meeting minutes, November 23, 1998).2,3

The present study was designed to answer the following questions: Do students who are absent from the classroom when data are collected differ on a variety of demographic and behavioral variables from students captured in a classroom-based sample? If such differences exist, does weighting based on selfreported absenteeism provide an accurate estimate of risk behavior for the entire student population?

METHODS

To ensure demographic, socioeconomic, and academic diversity, we selected 13 schools from among New York City's 114 high schools. Schools were stratified by type (vocational or comprehensive) and rate of absenteeism and were randomly selected within these strata. Half of the schools within each category were included because they had high rates of absenteeism. Tenth graders were selected because most students at this level are too young to legally “drop out.” While this limited the study's generalizability to the entire high school population, it eliminated bias introduced by including older students who had left school.

Risk behaviors addressed included drug and alcohol use, absenteeism, academic failure, sexual activity, and weapon possession. Questions were drawn from earlier studies of adolescent risk behavior.1,4–6 The survey was piloted in 1997 and conducted in 1998.

A 4-stage data collection strategy was used, with each stage representing a subsequent step away from standard classroom-based survey protocols and requiring an increased level of intensity and associated resources. The design involved the assumption that increased effort in locating and surveying students would result in improved “capture” rates. The stages may be seen as a proxy for extent of absenteeism; students with frequent absences are less likely to be surveyed when a 1-day capture technique is used.

The 4 stages were implemented sequentially without overlap. Stages 1 and 2 were classroom based and replicated methods used in previous studies. Stages 3 and 4 took place within the schools but outside of classrooms, incorporated financial incentives, and represented a substantial departure from previous research in the field.

During stage 1, all students present in the preselected required subject class were surveyed. Students entering the classroom late were surveyed in the following stage as if they had completed the survey with those attending class. Stage 2 involved another 1-day classroom-based sampling of only those students who were not in attendance during stage 1.

After completion of stage 2, “incentives” (i.e., gift certificates that could be used at local stores) were distributed to all students who had completed the survey; to maintain consistency with methodologies of national studies, we did not inform students of the incentives at the start of the study. Distribution of incentives after stage 2 did not affect the response rate up to that point, and it drew attention to the survey among those who skipped classes or school and who were therefore the kinds of students we were most interested in surveying in stages 3 and 4.

During stage 3, conducted over a 1-week period, notices were posted throughout each school building notifying eligible students of the incentive available for survey completion. In the final stage of the survey, stage 4, letters were mailed to the homes of all nonrespondents.

As a means of differentiating respondents from nonrespondents, survey responses were confidential, but surveys were not completed anonymously. The trained data collectors who administered the survey reaffirmed its confidential nature. Both student and parental consent were required before survey administration. Students were asked to take consent forms home for their parents or guardians; approximately 4% of parents–guardians refused to allow their children to participate. Students surveyed in stages 1 and 2 were compared with those surveyed in stages 3 and 4 (Table 1) in regard to demographic characteristics and risk-taking behaviors (e.g., frequency of alcohol and drug use, sexual activity, number and nature of school absences, and grades). In accordance with the methods used in some earlier studies, student responses for stages 1 and 2 were then weighted according to selfreported frequency of absence. These weighted estimates of risk behavior, including only students from stages 1 and 2, were compared with the estimates obtained when students from all 4 stages were included.

TABLE 1.

—Demographic Characteristics and Risk Behavior, by Survey Stage and Weighted by Self-Reported Absences: New York City, 1998

Characteristic Stages 1 & 2 (n = 1921), % Stages 3 & 4 (n = 128), % Significance (χ2) Stages 1 & 2 (Weighted), % Stages 1–4 (n = 2049), %
Sex .940
    Female 53 52 53 53
    Male 47 48 47 47
Age, y .017
    16 or younger 88 80 87 87
    17 or older 12 20 13 13
Ethnicity .034
    African American 30 37 30 30
    Caribbean American 25 30 23 25
    Asian 6 3 6 6
    Hispanic–Latino 29 21 30 28
    White (including “other”) 9 8 10 9
Household composition .333
    Both parents 32 20 31 31
    Other relative 10 19 10 10
    Single parent 59 62 60 59
No. of nonlegitimate absences .001
    0 70 56 68 69
    1–3 21 29 22 22
    4 or more 9 15 10 9
No. of classes skipped .000
    0 38 23 36 37
    1–5 47 54 48 47
    6 or more 15 23 16 15
Grades .002
    A's or B's 36 22 34 35
    C's 39 44 39 39
    D's or F's 26 34 28 25
Cigarette use in previous 4 weeks 16 23 .033 17 16
Marijuana use in previous 4 weeks 17 21 .224 18 17
Alcohol use in previous 4 weeks 36 37 .861 37 36
Sexually active 51 71 .000 52 52
Involved in HIV high-risk behavior sometimes or often 14 22 .015 14 14
Carried weapons to school 1 or more days 14 18 .183 14 14

Note. The weighting protocol is based on the hypothesis that a student who is in class a proportion of times over the school year has a relative chance or probability (p) of being present in class on the day the survey is administered. Accordingly, this student's response to a particular item on the survey should receive a weight of 1/p in computing the population estimates of this item for the group sampled in the single-stage sampling plan. If a student is in class 15 days out of 20, p = 15/20 or 0.75. The inverse of that proportion is 1.33, so the student's response would be weighted by 1.33. If a student is in class for 5 out of 20 days, the weight is 4.

RESULTS

A total of 2049 students, drawn from an initial sample of 2675, participated in the study (77% of potential respondents). Of those taking part, 1921 (94%) were surveyed during stages 1 and 2. The intensive efforts and resources of stages 3 and 4 yielded only 128 additional students; 23% of the initial respondents were never interviewed.

The data indicate that the stage at which a student was surveyed was a strong proxy for attendance (Table 1). For example, students surveyed in stages 1 and 2 were more likely than those surveyed in the later stages to report never having had a nonlegitimate absence (70% vs 56%; P < .01). Students surveyed in the first 2 stages were also far more likely to report never having skipped (45% vs 30%; P < .01). Students surveyed in stages 3 and 4 were more likely to be older, to be African or Caribbean American, and to have weaker grades.

Cigarette use was more common among those with poor attendance records (23% vs 16%; P < .05), but there were no statistically significant differences between groups in marijuana and alcohol use. Both groups reported negligible use of drugs other than marijuana and alcohol. Rates of sexual activity differed according to survey stage. For example, 71% of students surveyed in stages 3 and 4 reported sexual activity, as compared with 51% of those interviewed in stages 1 and 2 (P < .01).

Comparisons of risk behavior estimates obtained by weighting the data for students interviewed in stages 1 and 2 and estimates obtained by combining the 4 stages revealed very small differences (typically, only 1 percentage point). That is, because of the relatively small number of students surveyed in stages 3 and 4, the data produced by the more intensive survey strategies did not result in markedly changed estimates of risk behavior.

DISCUSSION AND CONCLUSIONS

In this study, we addressed 2 broad questions. First, do students absent from classrooms when data are collected differ, demographically or behaviorally, from students captured in classroom-based samples? The answer is yes. Our findings further suggest that students who are not included in classroom-based surveys are not well represented by students who are included. Intensive efforts to find such students, however, proved of limited benefit in improving the yield.

Second, given the differences between students typically and rarely surveyed, does weighting based on self-reported absenteeism provide an accurate estimate of risk behavior for all students? Including students from stages 3 and 4 did not significantly alter estimates of risk behavior, because we were unable to reach many of these students. Weighting the data for absenteeism marginally improved our estimates.

Clearly, the risk behaviors of students who attend school regularly and those who are frequently absent differ. In the case of New York City public high schools, the marginal improvements in estimates that were achieved by attempting to capture absent students through incentives and postal follow-ups, as opposed to a simple weighting procedure, did not merit the financial cost and effort required. One must be cautious in generalizing our results to other settings; however, we believe that the present findings are likely to hold true in other urban schools.

Acknowledgments

This research was funded by a grant from the National Institute on Drug Abuse (1R01DA09549-01A1). The study was approved by New York University's Institutional Review Board and the New York City Board of Education's Proposal Review Committee.

We acknowledge the hard work of our field team, including Linnea Carlson, Katherine Singh, and Gabriel Stover; the helpful guidance of our advisory team, Blanche Frank, Patrick O'Malley, and David Ward; and the cooperation of the administrators and teachers in the New York City school system and the participating students.

All of the authors contributed to the conception and design of the study and to the writing and revision of the paper.

Peer Reviewed

References

  • 1.Handbook for Conducting Youth Risk Behavior Surveys. Atlanta, Ga: Centers for Disease Control and Prevention; 1993.
  • 2.Johnston LD, O'Malley P.M. Issues of validity and population coverage in student surveys of drug use. In: Rouse BA, Kozel NJ, Richards LG, eds. Self-Report Methods of Estimating Drug Use: Meeting Current Challenges to Validity. Rockville, Md: Alcohol, Drug Abuse, and Mental Health Administration; 1985. NIDA Research Monograph 57. [PubMed]
  • 3.Rainone G, Marel R, Aryan Z, et al. Youth at Risk in New York State: Alcohol and Drug Use Findings. Albany, NY: New York State Office of Alcoholism and Substance Abuse Services, Bureau of Applied Studies; 1993.
  • 4.Institute for Social Research. Monitoring the Future: A Continuing Study of America's Youth. Ann Arbor, Mich: University of Michigan; 1997.
  • 5.Periodic Assessment of Drug Use Among Youth. Albany, NY: New York State Office of Alcoholism and Substance Abuse Services; 1996.
  • 6.New York City Condom Availability Study. New York, NY: New York University, Department of Health Studies; 1994.

Articles from American Journal of Public Health are provided here courtesy of American Public Health Association

RESOURCES