Abstract
Background
The European School Survey Project on Alcohol and Other Drugs (ESPAD) is a survey study that collects comparable data on substance use of students aged 15 to 16 years old in European countries. The present study aims at investigating the impact of school refusal to participate in ESPAD on substance use prevalence estimates.
Methods
Data came from the 2007 German ESPAD study; the sample consisted of 12,246 students in 552 schools within seven German federal states. A simulation approach was used in order to study the effects of systematic exclusion of participating schools on prevalence estimates of key ESPAD outcomes including the use of tobacco, alcohol, cannabis, and other illegal drugs.
Results
The systematic exclusion of schools based on city-, school-, and class size, school environment, and schools’ substance use policies resulted in significant changes in prevalence estimates in 23 of 25 examined combinations of selection criterion and outcome. Yet, these effects were small, with differences remaining below 3 percentage points around the original estimates.
Conclusions
This simulation approach suggests that nonparticipation of schools in surveys on students’ substance use in Germany does not largely affect the validity of resulting prevalence estimates. Even a reduced number of schools may be sufficient to gain valid prevalence figures.
Keywords: nonresponse, substance use, prevalence estimation, survey research, school, simulation
Introduction
School surveys are frequently used to collect data on substance use in student populations. The European School Survey Project on Alcohol and Other Drugs (ESPAD) is a study that collects comparable data on substance use of students aged 15 to 16 years old in European countries. The survey was first carried out in 26 countries in 1995, while the most recent survey collected data from 36 countries in 2011 (Hibell et al., 2012).
In most ESPAD countries a large proportion of sampled schools took part in the surveys. In the 2011 data collection, the school participation rate was 90% or higher in 22 out of the 36 countries and the average for all countries was 85% (Hibell et al., 2012). However, the average has decreased over the years from about 95% in the 1995 and 1999 data collections, dropping to 91% in 2003, 89% in 2007, and 85% in 2011 (Hibell et al., 2004, 2009, 2012). In 2011, the lowest participation rate was reported for Great Britain (6%), and three more countries (Denmark, Germany, and Norway) reported rates below 50%. Germany is the country with the most dramatic drop in the proportion of participating schools from 91% in 2003 and 2007 to only 40% in 2011 (Hibell et al., 2004, 2009, 2012). This makes Germany an interesting country to study the potential effects of decreasing school participation on substance use prevalence estimates.
High response rates have long been considered a key criterion for epidemiological studies, but a general trend of declining response rates has been reported in recent decades (Galea & Tracy, 2007). The central concern associated with low response rates is that the resulting estimates may be affected by nonresponse bias, which is introduced when participant characteristics associated with the nonresponse are also related to the outcome examined. Therefore, the results of studies with a low response rate may not be valid.
The potential consequences of unit nonresponse have predominantly been examined in studies sampling individuals (Gerrits, Van Den Oord, & Voogt, 2001; Groves & Peytcheva, 2008; Keeter, Miller, Kohut, Groves, & Presser, 2000; Shahar, Folsom, Jackson, & The Atherosclerosis Risk in Communities (ARIC) Study Investigators, 1996) or organizations with respondents on the organizational level (e.g., sampling of schools, survey among principals) (Kano, Franke, Afifi, & Bourque, 2008). This holds also true for the area of substance abuse research, where studies examining the effects of nonresponse are, to our knowledge, exclusively based on individuals (Kypri, Stephenson, & Langley, 2004; Lahaut et al., 2003; Lahaut, Jansen, Mheen, & Garretsen, 2002; Zhao, Stockwell, & Macdonald, 2009). While some of these studies concluded that nonresponse bias is a significant problem (Lahaut et al., 2002; Zhao et al., 2009), others reported the opposite (Kypri et al., 2004). On the other hand, the effects of nonparticipation of institutions (e.g., schools) on substance use prevalence estimates derived from individual’s responses nested within these institutions are unknown. Analysing data on school level provides the opportunity to examine the impact of nonparticipation related to structural, political, and social factors on the institutional rather than the individual level.
Objective of the current study and hypotheses
This study aimed at investigating the potential impact of school refusal to participate on substance use prevalence estimates based on 2007 ESPAD data collected in Germany. A simulation approach was used in order to study the effects of systematic exclusion of participating schools on the ESPAD key variables tobacco use, alcohol use, episodic heavy drinking (EHD), cannabis use, and use of illegal drugs other than cannabis.
The systematic selection and simulation of nonparticipation based on school characteristics were assumed to substantially influence substance use prevalences. For example, since we expected higher rates of substance use in larger cities (Tretter & Kraus, 2004), nonparticipation of schools from larger cities were expected to reduce prevalence estimates (Hypothesis 1). In addition, we explored if nonparticipation based on school or class size leads to substantial changes in substance use prevalence estimates.
Neighbourhood disadvantage may be related to adolescent substance use behaviours such as early initiation of substance use (Fite, Wynn, Lochman, & Wells, 2009) and a higher risk for cigarette, alcohol, or illegal drug use (Galea, Nandi, & Vlahov, 2004; Winstanley et al., 2008). Excluding schools from disadvantaged neighbourhoods was therefore hypothesized to lead to a decrease in prevalences (Hypothesis 2).
Lastly, school substance use policies and enforcement of these policies may be effective in reducing cigarette (Piontek, Bühler, & Kröger, 2007) or alcohol use (Evans-Whipp, Plenty, Catalano, Herrenkohl, & Toumbourou, 2013). We therefore expected that nonparticipation of schools with non-restrictive substance use policies to result in a reduction in prevalence estimates regarding cigarette and alcohol use (Hypothesis 3).
Methods
Procedure
Data from the 2007 ESPAD study in Germany were used (Kraus, Pabst, & Steiner, 2008). Seven out of 16 German federal states agreed to participate in the study, indicating a fairly good geographic representation of Germany: (1) Berlin as the capital within Brandenburg (2) as one of the three East-German states from the former German Democratic Republic, (3) Mecklenburg-Western Pomerania (northeast), (4) Thuringia (southeast), (5) Bavaria, a large rural state in the south, (6) Hesse, centrally located, and (7) Saarland (west). The target population consisted of students attending regular secondary schools in grades 9 and 10 (typical age range of students: 14-16 years). The sample was drawn using a stratified systematic sampling design with classes as primary sampling units (Hibell et al., 2009). First, classes of all schools within each participating state were stratified by grade and education level and the required sample size was calculated proportionally: Hauptschulen (low education), Realschulen (intermediate education), Gymnasien (high education), and Gesamtschulen (intermediate and high education). Classes within each stratum were sorted in ascending order according to community size. Second, the number of students within classes was listed consecutively, such that each student was given the same probability of being sampled. An interval “x” was calculated for each stratum based on the total number of students divided by the required number of students. Finally, each “xth” student representing one class was systematically selected. This sampling procedure resulted in 625 selected schools that were contacted and of these 567 (91%) participated. Overall student-level response rate was 80.6%.
Ethical approval for the German ESPAD study was obtained from the review board of the German Psychological Society (DGPs). Schools received information material on the purpose of the survey, student questionnaires, and a questionnaire assessing school characteristics to be completed by teachers of the participating classes. Students were informed that participation was voluntary and their parents had to provide informed consent. Student questionnaires did not contain any personal information on the identity of individuals and were completed during class hours. Reliability and validity of the ESPAD student questionnaire have been demonstrated and are reported elsewhere (Hibell et al., 2009).
Study sample
The original dataset contained information on 567 schools and 12,568 students of which 120 individual records with missing values on year of birth, gender, and more than 50% missing values were excluded (Kraus et al., 2008). An additional 15 schools were excluded since they did not complete the school characteristics questionnaire. The analytical sample consisted of 552 schools and 12,246 students, which reflects 88% of all 625 schools that were originally contacted.
Measures
School-level variables
All variables for simulating school refusal except for class size were part of the school characteristics questionnaire completed by the respective supervising teacher.
City size
City size was assessed with the question “How many inhabitants live in the city in which the school is located?” and responses were recorded on an eight-point-scale (“less than 2000” to “more than 1 million”). A median splits was used to dichotomize this variable (small: cities < 20,000 inhabitants; large: cities ≥ 20,000 inhabitants) for subsample selection.
School size
School size was assessed with the question “How many students go to your school?” and responses were coded on a six-point-scale (“less than 200” to “more than 1000”). Again, the selection of subsamples according to school size was based on a median split (small: schools ≤ 600 students; large: schools > 600 students).
Class size
Class size was calculated from the number of individual questionnaires returned by students. A median split was calculated (small: classes ≤ 21 students; large: classes > 21 students) to select subsamples.
School environment
Three items were used to measure school environment (e.g., “Everything in the close surroundings is orderly and well-kept.”). Response options were on a five-point-scale from “applies” to “does not apply”. A mean score was calculated as an indicator of school environment (Cronbach’s α = .56). The selection subsamples according to school environment was based on a median split (disadvantaged: neighbourhood score ≤ 2.5; non-disadvantaged: neighbourhood score > 2.5).
School substance use policies
Four items measured school substance use policies regarding students’ smoking and alcohol use. Teachers indicated whether their school had a ban on smoking and alcohol use and if there were sanctions in place in case these policies were violated. Schools were coded as having restrictive school policies if they contained both substance use bans and sanctions, and were otherwise coded as having non-restrictive policies.
Student-level variables
Smoking
Survey respondents were asked about their frequency and quantity of cigarette smoking in the past 30 days on a seven-point-scale (“not at all” to “more than 20 cigarettes per day”). This variable was dichotomized to indicate any smoking in the past 30 days.
Alcohol use
Alcohol use in the past 30 days was assessed by a beverage-specific quantity-frequency index for beer, wine, sprits, and alcopops (Gmel & Rehm, 2004; Hibell et al., 2009). Episodic heavy drinking (EHD) was assessed as drinking five or more alcoholic drinks on one occasion within the past 30 days. Both variables were dichotomized to indicate any alcohol use or EHD in the past 30 days.
Cannabis and other illegal drug use
Participants indicated the frequency of cannabis use within the past 12 months as well as the frequency of lifetime use of other illegal drugs (ecstasy, amphetamines, LSD, crack, cocaine, heroin, GHB, magic mushrooms). Illegal drug use other than cannabis was coded if any use of the above listed substances was reported. Both variables were dichotomized to indicate cannabis use in the past 12 months or lifetime use of illegal drugs other than cannabis (other illegal drug use).
Statistical analyses
A simulation approach was used to create scenarios with different participation rates and to compare the prevalences of substance use in the resulting restricted samples with the original sample. First, prevalence estimates of the different substance use outcome measures were aggregated by school. Rates of missing data on substance use among students ranged from 0.03% (cigarette use) to 3.9% (episodic heavy drinking) and were handled via case wise deletion. Since the analysed sample of 552 schools represented 88% of all contacted school in the 2007 German ESPAD survey, it was assumed to represent an unbiased sample and was used as baseline (original sample). Subsamples were selected based on the following scenarios: (1) Selection of schools in large cities; (2) Selection of large schools; (3) Selection of large classes; (4) Selection of schools with disadvantaged school environment; and (5) Selection of schools with non-restrictive substance use policies. For scenarios 1-4, median splits were used to dichotomize the selection variables; for scenario 5, schools without consistent substance use bans and enforcement were selected (see measures). Second, the sample size was reduced in incremental steps to 95%, 90%, and in 10% increments down to 40% of the initial size (restricted samples). In each step, 10 randomly drawn subsamples were used and averages of mean prevalence as well as 95% confidence intervals of all outcome variables were calculated. Confidence intervals were based on standard errors corrected for sampling from a finite population (Cochran, 1977). Statistical significance between prevalence estimates was assessed by examining the confidence intervals of the selected and the excluded sample and significance was assumed when the confidence intervals did not overlap. In order to judge the magnitude of resulting changes in prevalence estimates we conducted a series of power analyses with the software GPower 3.1 (Faul, Erdfelder, Lang, & Buchner, 2007) to examine if the largest differences between full and reduced sample for every substance would be significant, if we assumed a sample size of 2,500 students. This sample size was chosen, since a sample of more than 2,400 students was recommended to participating countries in the 2007 ESPAD study (Hibell et al., 2009).
Since this study employed a simulation approach based on data from an empirical study, the variables used for selection of schools cannot be assumed as independent. Pairwise Pearson correlations between selection variables on the school level were calculated in order to assess their potential inter-dependence. In order to estimate the contribution of the different school-level selection variables to the substance use outcome prevalences, additional multiple regression analyses were conducted. All school selection variables were included simultaneously as predictors.
Results
Sample description and correlations between school-level predictors
Sample characteristics of students and schools are summarized in Table 1. The highest prevalence was reported for past 30-day alcohol use (82.5%), followed by past 30-day EHD (60.1%), 30-day cigarette use (38.6%), 12-month cannabis use (17.7%), and lifetime use of illegal drugs other than cannabis (11.5%). Pairwise correlations of the school-level predictors are shown in Table 2. City, school, and class size were all significantly positively correlated with each other. Furthermore, city size was significantly negatively correlated with school environment, indicating that the quality of school environment decreases with increasing city size. Lastly, school substance use policies were negatively correlated with both school and class size suggesting that smaller schools and smaller classes had more restrictive substance use policies.
Table 1.
Sample description
| Variable | M (SD)/ n (%) | |
|---|---|---|
| Students (n=12,246) | ||
| Age | 15.66 (0.83) | |
| Gender female | 50.9% | |
| Prevalence cigarettes 30 days | 38.6% | |
| Prevalence alcohol 30 days | 82.5% | |
| Prevalence episodic heavy drinking 30 days | 60.9% | |
| Prevalence cannabis 12 months | 17.7% | |
| Prevalence illegal drugs other than cannabis lifetime | 11.5% | |
| Schools (n=552) | ||
| City size | ||
| Large | 250 (45.3%) | |
| Small | 277 (50.2%) | |
| Missing | 25 (4.5%) | |
| School size | ||
| Large | 239 (43.3%) | |
| Small | 293 (53.1%) | |
| Missing | 20 (3.6%) | |
| Class size | ||
| Large | 287 (52.0%) | |
| Small | 265 (48.0%) | |
| Missing | - | |
| School environment | ||
| Disadvantaged | 217 (39.3%) | |
| Not disadvantaged | 314 (56.8%) | |
| Missing | 21 (3.8%) | |
| School substance use policies | ||
| Non-restrictive | 197 (35.7%) | |
| Restrictive | 334 (60.5%) | |
| Missing | 21 (3.8%) | |
Table 2.
Pairwise correlations between school selection variables
| City size | School size | Class size | School environment1 |
School policies2 |
|
|---|---|---|---|---|---|
| City size | 1 | ||||
| School size | 0.22*** | 1 | |||
| Class size | 0.10* | 0.46*** | 1 | ||
| School environment1 |
−0.32*** | −0.03 | 0.02 | 1 | |
| School policies2 |
−0.00 | −0.18*** | −0.12** | −0.00 | 1 |
Note:
higher values indicating more positive school environment
coded as 0 non-restrictive substance use policies and 1 restrictive school substance use policies
p<.05
p<.01
p<.001.
Multiple regression results
Multiple regression analysis results showed that city size was a significant predictor of all outcomes but other illegal drug use, when all other school-level variables were controlled for (Table 3). In addition, school size predicted cigarette use, EHD, and other illegal drug use, while class size only predicted cigarette use. School environment predicted alcohol use, EHD, and cannabis use, whereas school policies did not predict any substance use outcome.
Table 3.
Results of the multiple linear regression analyses with school selection variables predicting the different substance use outcomes
| Predictors | Outcomes |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Cigarette use | Alcohol use | Episodic heavy drinking |
Cannabis use | Illegal drug use other than cannabis |
||||||
|
|
||||||||||
| Coefficient (SE) |
t | Coefficient (SE) |
t | Coefficient (SE) |
t | Coefficient (SE) |
t | Coefficient (SE) |
t | |
|
|
||||||||||
| City size | −0.011 (0.004) |
−2.9** | −0.026 (0.003) |
−9.4*** | −0.031 (0.004) |
−8.5*** | 0.010 (0.003) |
3.7*** | −0.003 (0.002) |
−1.4 |
| School size | −0.029 (0.006) |
− 5.3*** |
−0.003 (0.004) |
−0.7 | −0.016 (0.005) |
−3.1** | −0.000 (0.004) |
−0.1 | −0.013 (0.003) |
−4.2*** |
| Class size | −0.005 (0.002) |
−3.0** | 0.002 (0.001) |
1.6 | −0.001 (0.001) |
−0.7 | −0.002 (0.001) |
−1.7 | −0.001 (0.001) |
−1.6 |
| School environment1 |
0.010 (0.010) |
0.9 | 0.037 (0.007) |
5.1*** | 0.024 (0.009) |
2.6* | 0.016 (0.007) |
2.2* | 0.007 (0.006) |
1.3 |
| School policies2 |
0.026 (0.015) |
1.7 | −0.012 (0.011) |
−1.2 | 0.016 (0.014) |
1.1 | −0.002 (0.011) |
−0.2 | −0.006 (0.008) |
−0.6 |
| R-square | 16.5% | 25.6% | 22.2% | 3.3% | 8.0% | |||||
Note:
higher values indicating more positive school environment
coded as 0 - non-restrictive substance use policies and 1 - restrictive school substance use policies
p<.05
p<.01
p<.001.
Simulation results
Figures 1 to 5 show the results of the simulated reduction in sample size. The corresponding sample sizes are displayed in Table 4. When excluding schools based on city size, particularly the prevalences of cigarette use, alcohol use, and EHD were affected, with the first increasing up to 1.0 percentage point (pp) and the latter two increasing by around 2.0 pps, which were among the largest changes following simulated selective dropout. Prevalence of cannabis use was decreased by up to 1.0 pp and prevalence of other illegal drug use was increased by 0.3 pps when excluding schools from larger cities. Simulated dropout regarding large schools caused cigarette use prevalence to increase by 2.0 pps, EHD prevalence by 1.4 pps and other illegal drug use by 0.9 pps. Alcohol and cannabis use were largely unaffected with prevalence changes of less than 0.3 pps. When excluding schools based on class size, cigarette use prevalence increased by about 1.8 pps, EHD prevalence by 1.0 pp, and prevalence of other illegal drug use by 0.8 pps; alcohol and cannabis use prevalences remained largely the same as in the original sample with differences of 0.3 pps or less. Simulated dropout based on schools with disadvantaged environment resulted in prevalence increases of 1.2 pps in alcohol use and 1.0 pp in EHD. Cigarette use prevalence was increased by 0.7 pps and other illegal drug use and cannabis use prevalences were least affected (0.3 and 0.2 pps, respectively). Of all variables that were used to simulate systematic school dropout, non-restrictive school policy had the smallest effect overall: Prevalences of cigarette use and EHD increased by 0.7 pps and by 0.4 pps, respectively, and differences for the other substance use prevalences remained at or below 0.3 pps.
Figure 1.
Simulation results for past 30-day prevalences of cigarette use and 95% confidence intervals according to sample size for systematic selection of schools
Note: Prevalence on the y-axis, sample size on the x-axis; * indicates significant difference between selected and excluded samples.
Figure 5.
Simulation results for lifetime prevalences of illegal drug use other than cannabis and 95% confidence intervals according to sample size for systematic selection of schools
Note: Prevalence on the y-axis, sample size on the x-axis; * indicates significant difference between selected and excluded samples.
Table 4.
Sample size reduction
| Sample size (% of original sample) | ||||||||
|---|---|---|---|---|---|---|---|---|
|
| ||||||||
| Selection variable | 100% | 95% | 90% | 80% | 70% | 60% | 50% | 40% |
| City size large (n) | 552 | 540 | 527 | 502 | 477 | 452 | 427 | 402 |
| School size large (n) | 552 | 540 | 528 | 504 | 480 | 456 | 433 | 409 |
| Class size large (n) | 552 | 538 | 523 | 495 | 466 | 437 | 409 | 380 |
| School environment disadvantaged (n) |
552 | 541 | 530 | 509 | 487 | 465 | 444 | 422 |
| School substance use policies non-restrictive (n) |
552 | 542 | 533 | 513 | 494 | 474 | 455 | 435 |
Overall, the small confidence intervals in Figures 1 - 5 indicate that systematic dropout resulted in prevalence estimates significantly different from those found in the original sample (the only exceptions were cannabis use prevalence in case of the selection variables school size and school substance use policies, where no significant differences were observed). In order to estimate the magnitude of each of the resulting changes in prevalence estimates we conducted a series of power analyses. With an assumed standard sample size of 2,500 students, none of the differences in prevalence between original and the restricted samples reached statistical significance.
Discussion
The present study investigated the potential effects of school nonparticipation on prevalence estimates regarding the use of tobacco, alcohol, cannabis, and illegal drugs other than cannabis, using data from the German 2007 ESPAD study.
The simulation of systematic nonparticipation of schools resulted in significant changes of prevalence estimates in 23 out of 25 possible combinations of outcomes and selection variables. However, the changes in prevalence estimates were generally small. For example, prevalence estimates of cigarette use, alcohol use, and EHD increased between 1.0 and 2.2 pps, when up to 60% of schools from large cities were excluded from the sample. The fact that prevalences increased rather than decreased when schools from large cities were excluded was unexpected and contradicted Hypothesis 1. With regard to alcohol use these findings may be due to adolescents with a migration background from abstinence oriented drinking cultures that drink less (Lampert & Thamm, 2007) and predominantly reside in larger cities (Statistisches Bundesamt, 2014). Indeed, the 2007 ESPAD study reported less drinking among students in Berlin, the capital and largest German city (Kraus et al., 2008).
Excluding large schools or large classes from the sample resulted in an increase in the prevalences of smoking and EHD. Since peer substance use is one of the most important factors associated with adolescents’ substance use (Hawkins, Catalano, & Miller, 1992; Simons-Morton, 2007) there may be an increased contagion effect in small schools or classes. With regard to EHD, this effect can potentially be explained by the high correlations of school and class size in our data. Indeed, as observed in the multiple regression analyses, class size was not a significant predictor of EHD, when the other school-level variables were accounted for.
Furthermore, exclusion of schools based on the school environment was found to affect all assessed substance use outcomes. Interestingly, prevalence estimates increased when schools with disadvantaged environment were successively excluded. These findings do not support our Hypothesis 2 and contradict results from previous studies reporting increases in early substance use initiation and higher substance use in general in disadvantaged neighbourhoods (Fite et al., 2009; Galea et al., 2004; Winstanley et al., 2008). With regard to alcohol use and EHD this finding may reflect that students in schools with more advantaged neighbourhoods may be from a higher socioeconomic background and therefore may have more expendable income, which has been found to be associated with elevated levels of alcohol use (Bellis et al., 2007). However, it should also be noted that school environment in our study was based on self-reports from teachers of participating schools, whereas previous studies used self-reports of adolescents, who reported on the neighbourhood they live in (Winstanley et al., 2008). School environment may not necessarily be identical to the environment students live in. Lastly, the conducted multiple regression analyses suggested that school environment did not significantly predict cigarette and other illegal drug use prevalences, so at least for these two outcomes the association between school environment and substance use is likely due to other correlated school-level factors such as city size.
Cannabis use prevalence estimates were not strongly affected by simulated school dropout with changes of 0.2 pps or less. The exception was a decreasing trend of up to 0.8 pps when schools from large cities were excluded. This finding is consistent with previous research reporting higher rates of cannabis use in urban compared to rural environments (Ousey & Maume, 1997; Tretter & Kraus, 2004) and may reflect easier access to cannabis in cities or social norms that are more conducive to cannabis use.
The prevalence of illegal drug use other than cannabis increased slightly when large schools or large classes were excluded (0.8 – 0.9 pps), but were not strongly affected by simulated nonparticipation based on other variables (changes up to 0.3 pps). While we do not have a conclusive explanation for these findings, it may be the case that for other illegal drugs, the uniform legal context across all of Germany does not allow for a large variance in structural and political factors that may be associated with their use.
Lastly, substance use prevalences remained largely unchanged when schools with non-restrictive substance use policies were excluded, failing to support our Hypothesis 3. The only exception was a 0.2 pps decrease observed for alcohol use. Yet, even small increases in prevalence were observed in the case of cigarette use (0.7 pps) and EHD (0.4 pps). This finding contrasts with previous studies reporting that school policies and enforcement may have protective effects on smoking and alcohol use (Evans-Whipp et al., 2013; Piontek et al., 2007). Again, this finding is likely due to the correlation between school-level variables, since larger schools reported more non-restrictive substance use policies and the exclusion of larger schools also resulted in an increase in prevalence estimates regarding cigarette use and EHD. The multiple regression analyses support this explanation, since school policies did not predict substance use prevalences over and above the other school-level variables. Given the cross-sectional nature of our data it should further be noted that we cannot draw conclusions regarding the effectiveness of school-level substance use policies.
Overall, our findings are in line with the results from previous epidemiological studies, which reported that the consequences of unit nonresponse are relatively small (Galea & Tracy, 2007; Gerrits et al., 2001; Keeter et al., 2000; Shahar et al., 1996) and may not considerably bias prevalence estimates (Kypri et al., 2004). Our study adds to these findings by expanding the sampling frame of individuals to schools. The robustness of estimates independently of school nonresponse of any kind (city-, school-, and class size, school environment, and schools’ substance use policies) indicates that these factors are associated with individual students’ substance use behaviours to a lesser extent than expected. Or stated differently, students’ substance use behaviour at different education levels seems to be rather similarly distributed across city-, school-, and class size, as well as across schools with different environments and various substance use policies. For Germany, we thus conclude that smaller sample sizes by education level may be sufficient in estimating substance use behaviour. Taken together, our findings suggest that nonparticipation of a substantial number of schools may not seriously bias the prevalence rates obtained from a student survey focusing on substance use behaviour.
Limitations
The findings of the present study should be interpreted with several limitations in mind. Firstly, we conducted a simulation study based on a single cross-sectional study in 7 German federal states. Results may not be generalizable to the rest of Germany, other countries, or other ESPAD study waves. Selection variables on the school level were correlated with each other and were not experimentally controlled; therefore, some associations between school selection variables and substance use prevalences may be in part due to third school-level variables and causality should not be inferred. Moreover, the sample of 552 schools was assumed to represent an unbiased original sample, but since this sample represented only 88% of schools which participated in 2007, we cannot rule out that the missing 12% may have already introduced bias to the data. Furthermore, self-reports of students were used for all measures of substance use and self-reports of teachers to assess school characteristics. Self-reports may be prone to recall bias and under- or over-reporting. Specifically, the measure of school environment had low internal consistency. Lastly, due to differences in sample size across the selection variables, results of the different simulations are limited in their comparability.
Conclusions
Systematic school-level nonresponse was associated with significant changes in prevalence estimates based on the 2007 German ESPAD study. However, our results suggest that these changes remain small, with the largest effects being increases in prevalence rates of approximately 2 pps. Overall, schools and school classes seem to represent the distribution of substance use among adolescents in the general population fairly adequately. Our results indicate that school nonparticipation in surveys that aim at assessing substance use among students may not be as worrisome as expected with regard to the validity of resulting prevalence estimates. Future research needs to corroborate our findings using samples from other countries.
Highlights.
School nonresponse in a survey on students’ substance use (ESPAD study) was simulated
Systematic exclusion of schools resulted in significant changes of prevalences
The resulting changes were small overall
Even a reduced number of schools may be sufficient to gain valid estimates
Figure 2.
Simulation results for past 30-day prevalences of alcohol use and 95% confidence intervals according to sample size for systematic selection of schools
Note: Prevalence on the y-axis, sample size on the x-axis; * indicates significant difference between selected and excluded samples.
Figure 3.
Simulation results for past 30-day prevalences of episodic heavy drinking and 95% confidence intervals according to sample size for systematic selection of schools
Note: Prevalence on the y-axis, sample size on the x-axis; * indicates significant difference between selected and excluded samples.
Figure 4.
Simulation results for past 12-month prevalences of cannabis use and 95% confidence intervals according to sample size for systematic selection of schools
Note: Prevalence on the y-axis, sample size on the x-axis; * indicates significant difference between selected and excluded samples.
Acknowledgements
Funding of the project was provided by the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA): CC.13.SDI.021. The preparation of this manuscript was supported in part by the National Cancer Institute: NCI CA-113710.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Declaration of interests
The authors report no conflicts of interest.
References
- Bellis MA, Hughes K, Morleo M, Tocque K, Hughes S, Allen T, Fe-Rodriguez E. Predictors of risky alcohol consumption in schoolchildren and their implications for preventing alcohol-related harm. Substance Abuse Treatment, Prevention, and Policy. 2007;2(1):15. doi: 10.1186/1747-597X-2-15. doi:10.1186/1747-597X-2-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cochran WG. Sampling Techniques. 3rd ed. John Wiley & Sons; New York: 1977. [Google Scholar]
- Evans-Whipp TJ, Plenty SM, Catalano RF, Herrenkohl TI, Toumbourou JW. The impact of school alcohol policy on student drinking. Health Education Research. 2013;28(4):651–662. doi: 10.1093/her/cyt068. doi:10.1093/her/cyt068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007;39(2):175–191. doi: 10.3758/bf03193146. doi:10.3758/BF03193146. [DOI] [PubMed] [Google Scholar]
- Fite PJ, Wynn P, Lochman JE, Wells KC. The Influence of Neighborhood Disadvantage and Perceived Disapproval on Early Substance Use Initiation. Addictive Behaviors. 2009;34(9):769–771. doi: 10.1016/j.addbeh.2009.05.002. doi:10.1016/j.addbeh.2009.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Galea S, Nandi A, Vlahov D. The Social Epidemiology of Substance Use. Epidemiologic Reviews. 2004;26(1):36–52. doi: 10.1093/epirev/mxh007. doi:10.1093/epirev/mxh007. [DOI] [PubMed] [Google Scholar]
- Galea S, Tracy M. Participation Rates in Epidemiologic Studies. Annals of Epidemiology. 2007;17(9):643–653. doi: 10.1016/j.annepidem.2007.03.013. doi:10.1016/j.annepidem.2007.03.013. [DOI] [PubMed] [Google Scholar]
- Gerrits MH, Van Den Oord EJCG, Voogt R. An Evaluation of Nonresponse Bias in Peer, Self, and Teacher Ratings of Children’s Psychosocial Adjustment. Journal of Child Psychology and Psychiatry. 2001;42(5):593–602. doi:10.1111/1469-7610.00755. [PubMed] [Google Scholar]
- Gmel G, Rehm J. Measuring alcohol consumption. Contemporary Drug Problems. 2004;31(3):467–540. [Google Scholar]
- Groves RM, Peytcheva E. The Impact of Nonresponse Rates on Nonresponse Bias A Meta-Analysis. Public Opinion Quarterly. 2008;72(2):167–189. doi:10.1093/poq/nfn011. [Google Scholar]
- Hawkins J, Catalano R, Miller JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin. 1992;112(1):64–105. doi: 10.1037/0033-2909.112.1.64. doi:10.1037/0033-2909.112.1.64. [DOI] [PubMed] [Google Scholar]
- Hibell B, Andersson B, Bjarnason T, Ahlström S, Balakireva O, Kokkevi A, Morgan M. Alcohol and other drug use among students in 35 European countries. The Swedish Council for Information on Alcohol and other Drugs (CAN); Stockholm: 2004. The ESPAD report 2003. Retrieved from http://www.espad.org/en/Reports--Documents/ESPAD-Reports/ [Google Scholar]
- Hibell B, Guttormsson U, Ahlström S, Balakireva O, Bjarnason T, Kokkevi A, Kraus L. Substance use among students in 35 European countries. The Swedish Council for Information on Alcohol and other Drugs (CAN); Stockholm: 2009. The 2007 ESPAD report. Retrieved from http://www.espad.org/en/Reports--Documents/ESPAD-Reports/ [Google Scholar]
- Hibell B, Guttormsson U, Ahlström S, Balakireva O, Bjarnason T, Kokkevi A, Kraus L. Substance use among students in 36 European countries. The Swedish Council for Information on Alcohol and other Drugs (CAN); Stockholm: 2012. The 2011 ESPAD report. Retrieved from http://www.espad.org/en/Reports--Documents/ESPAD-Reports/ [Google Scholar]
- Kano M, Franke T, Afifi AA, Bourque LB. Adequacy of Reporting Results of School Surveys and Nonresponse Effects: A Review of the Literature and a Case Study. Educational Researcher. 2008;37(8):480–490. doi:10.3102/0013189X08326859. [Google Scholar]
- Keeter S, Miller C, Kohut A, Groves RM, Presser S. Consequences of Reducing Nonresponse in a National Telephone Survey. Public Opinion Quarterly. 2000;64(2):125–148. doi: 10.1086/317759. doi:10.1086/317759. [DOI] [PubMed] [Google Scholar]
- Kraus L, Pabst A, Steiner S. Europäische Schul̈erstudie zu Alkohol und anderen Drogen 2007 (ESPAD): Befragung von Schul̈erinnen und Schülern der 9. und 10. Klasse in Bayern, Berlin, Brandenburg, Hessen, Mecklenburg-Vorpommern, Saarland und Thüringen The 2007 European School Survey Project on Alcohol and other Drugs (ESPAD): Survey among pupils in grade 9 and 10 in Bavaria, Berlin, Brandenburg, Hesse, Mecklenburg-Western Pomerania, Saarland and Thuringia (No. 165) IFT Institut für Therapieforschung; München: 2008. Retrieved from http://www.ift.de/index.php?id=344. [Google Scholar]
- Kypri K, Stephenson S, Langley J. Assessment of Nonresponse Bias in an Internet Survey of Alcohol Use. Alcoholism: Clinical and Experimental Research. 2004;28(4):630–634. doi: 10.1097/01.alc.0000121654.99277.26. doi:10.1097/01.ALC.0000121654.99277.26. [DOI] [PubMed] [Google Scholar]
- Lahaut VMHCJ, Jansen HAM, Mheen D. van de, Garretsen HFL. Non-Response Bias in a Sample Survey on Alcohol Consumption. Alcohol and Alcoholism. 2002;37(3):256–260. doi: 10.1093/alcalc/37.3.256. doi:10.1093/alcalc/37.3.256. [DOI] [PubMed] [Google Scholar]
- Lahaut VMHCJ, Jansen HAM, Mheen D. van de, Garretsen HFL, Verdurmen JEE, Dijk A. van. Estimating Non-Response Bias in a Survey on Alcohol Consumption: Comparison of Response Waves. Alcohol and Alcoholism. 2003;38(2):128–134. doi: 10.1093/alcalc/agg044. doi:10.1093/alcalc/agg044. [DOI] [PubMed] [Google Scholar]
- Lampert T, Thamm M. Tabak-, Alkohol- und Drogenkonsum von Jugendlichen in Deutschland. Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz. 2007;50(5):600–608. doi: 10.1007/s00103-007-0221-y. doi:10.1007/s00103-007-0221-y. [DOI] [PubMed] [Google Scholar]
- Ousey GC, Maume MO. The grass is always greener: Explaining rural and urban differences in marijuana use. Sociological Focus. 1997;30(3):295–305. doi:10.1080/00380237.1997.10571080. [Google Scholar]
- Piontek D, Bühler A, Kröger C. Schulische Tabakprävention. Zur Wirksamkeit verhältnispräventiver Maßnahmen gegen das Rauchen von Jugendlichen Smoking prevention at schools. On the effectiveness of structural prevention activities on adolescent smoking. Prävention Und Gesundheitsförderung. 2007;2(2):129–135. doi:10.1007/s11553-007-0054-4. [Google Scholar]
- Shahar E, Folsom AR, Jackson R, The Atherosclerosis Risk in Communities (ARIC) Study Investigators The effect of nonresponse on prevalence estimates for a referent population: Insights from a population-based cohort study. Annals of Epidemiology. 1996;6(6):498–506. doi: 10.1016/s1047-2797(96)00104-4. doi:10.1016/S1047-2797(96)00104-4. [DOI] [PubMed] [Google Scholar]
- Simons-Morton B. Social influences adolescent substance use. American Journal of Health Behavior. 2007;31(6):672–684. doi: 10.5555/ajhb.2007.31.6.672. doi:10.5555/ajhb.2007.31.6.672. [DOI] [PubMed] [Google Scholar]
- Statistisches Bundesamt . Bevölkerung und Erwerbstätigkeit: Ausländische Bevölkerung - Ergebnisse des Ausländerzentralregisters 2013 [Population and employment: Foreign population - results from the Central Register of Foreigners] Statistisches Bundesamt; Wiesbaden: 2014. Retrieved from https://www.destatis.de/DE/Publikationen/Thematisch/Bevoelkerung/MigrationIntegration/AuslaendBevoelkerung.html. [Google Scholar]
- Tretter F, Kraus L. Stadtspezifische Prävalenz des Drogenkonsums und ihre UrsachenStadtspezifische Prävalenz des Drogenkonsums und ihre Ursachen [Prevalence and causes of drug use on city level] Sucht. 2004;50(1):5–7. [Google Scholar]
- Winstanley EL, Steinwachs DM, Ensminger ME, Latkin CA, Stitzer ML, Olsen Y. The association of self-reported neighborhood disorganization and social capital with adolescent alcohol and drug use, dependence, and access to treatment. Drug and Alcohol Dependence. 2008;92(1-3):173–182. doi: 10.1016/j.drugalcdep.2007.07.012. doi:10.1016/j.drugalcdep.2007.07.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhao J, Stockwell T, Macdonald S. Non–response bias in alcohol and drug population surveys. Drug and Alcohol Review. 2009;28(6):648–657. doi: 10.1111/j.1465-3362.2009.00077.x. doi:10.1111/j.1465-3362.2009.00077.x. [DOI] [PubMed] [Google Scholar]