Skip to main content
. 2022 Aug 29;2022(8):CD011677. doi: 10.1002/14651858.CD011677.pub3

McCormick 1995.

Study characteristics
Methods Trial name: The North Carolina School Health and Tobacco Education Project (SHTEP)/Skills Management and Resistance Training (SMART)
Study design: cluster‐RCT
Intervention duration: 4 years
Length of follow‐up from baseline: 4 years
Differences in baseline characteristics: not reported
Unit of allocation: school district
Unit of analysis: school district and individual teachers
Participants School type: junior high or middle schools
Region: North Carolina, USA
Demographic/socioeconomic characteristics: not reported
Inclusion/exclusion criteria: to be eligible for participation, a school district had to have ≥ 2 junior high or middle schools (51/140 school districts in North Carolina met this criterion). All teachers who were eligible to teach health.
Number of schools allocated
‐ 21 districts (11 intervention, 10 control)
‐ Initial implementation = 69 teachers (51 intervention, 18 control)
‐ Maintained implementation = 136 teachers (86 intervention, 50 control)
‐ 42 schools
Numbers by trial group
n (controls baseline) = 10
n (controls follow‐up) = 10
n (interventions baseline) = 11
n (interventions follow‐up) = 11
Recruitment
School districts:28 districts were randomly selected and a 5‐stage strategy was used to recruit districts to the study. After a school district agreed to participate, it was randomly assigned to either the experimental or control condition, resulting in 11 experimental and 10 control districts.
Schools:in both experimental and control school districts, ≥ 2 schools and ≥ 2 classrooms per school participated in the study
Teachers: all teachers in study districts identified by their schools as "eligible to teach health" were included in the sample.
Students: not reported
Recruitment rate
School districts:21 school districts of 28 identified, therefore, 75%
Schools:approximately 50 schools
Teachers at baseline 69/115 = 60%
Teachers at follow‐up 136/175 = 78%
Classrooms: approximately 100 classrooms
Students:approximately 3000 students exposed to 1 of the tobacco prevention curricula
Interventions Number of experimental conditions: 2 (1 intervention, 1 control)
Policies, practices or programmes targeted by the intervention
School health/tobacco prevention curricula (schools districts offered choice of 3: Growing Healthy, Teenage Health Teaching Modules, Project SMART).
Implementation strategies
EPOC: educational meetings
‐ Implementation intervention: in‐depth training for teachers and administrators on the use of the specific curriculum that had been adopted.
EPOC: educational materials
‐ Schools were provided with curricular materials (schools chose the most suited of 3 options).
EPOC: local consensus processes
‐ Signing of a Memorandum of Understanding: a Memorandum of Understanding, specifying which curriculum was adopted, how many lessons were to be taught, and how many classes would receive instruction, was signed by each district. Adoption intervention: process consultation for adoption. A consultation workshop was conducted with each experimental district to inform school personnel about the 3 health curricula that were being disseminated.
Theoretical underpinning: diffusion of innovation
Description of control: districts in the control conditions were mailed curricula materials and provided TA upon request, but did not receive the training sessions (EPOC: educational material).
Outcomes Outcome relating to the implementation of school policies, practices or programmes
‐ % later implementation of curriculum for school district
‐ Extent later implementation mean for school district (% of total curriculum activities taught)
Data collection method: each teacher who was eligible to teach health received an implementation check sheet and was asked to indicate which activities were taught. Data were also aggregated at school district level. Assessed at year 4 follow‐up "later implementation". A dichotomous measure of implementation (i.e. yes/no) necessary but not sufficient to assess the implementation of a school health curriculum. Therefore, Implementation check sheets were also used to assess implementation as the percentage of total curriculum activities that were taught (extent of implementation).
Validity of measures used: not reported/self‐report methods
Data collection method: 13‐item measure of awareness and concern among teachers and administrators about tobacco use among students
Validity of measures used: validity was not reported, although the authors reported the instrument was described elsewhere.
Outcome relating to cost: not reported
Outcome relating to adverse consequences: not reported
Outcome relating to child diet, PA or weight status: not reported
Notes Research funding: National Cancer Institute (#5 R01 CA 459907‐02).
Conflicts of interest: not reported
Risk of bias
Bias Authors' judgement Support for judgement
Random sequence generation (selection bias) Unclear risk Random sequence generation procedure not described.
Allocation concealment (selection bias) Unclear risk No information provided about allocation concealment and, therefore, it was unclear if allocation was concealed.
Blinding of participants and personnel (performance bias)
Implementation outcome High risk Outcome group: all
Given the nature of the intervention, participants and study personnel were likely to have been aware of study allocation and, therefore, high risk of performance bias.
Blinding of outcome assessment (detection bias)
Implementation outcome High risk Outcome group: all.
All outcomes subjective, self‐report.
There was no mention of blinding of participants and personnel. Teachers completed check sheets that assessed their delivery of curricula activities and researchers conducted site visits at implementing schools. High risk of performance bias as the implementation outcomes likely to be influenced by knowledge of group allocation.
Incomplete outcome data (attrition bias)
Implementation outcome High risk Outcome group: organisational climate.
The implementation measures represent cross‐sectional assessments. There was low attrition as only 1 experimental and 1 control district dropped out of the study (low risk of bias).
Outcome group: organisational size
Organisational size represented the total student enrolment for each school district and was obtained from North Carolina Department of Public Instruction records. The smallest district had 4134 students and the largest district had 29,532 students.
Intervention = 9 districts (82%)
Control = 7 districts (70%)
Imbalance across intervention and control district response (high risk of bias)
Outcome group: time of adoption
Time of adoption range 9.57–19.86 weeks
Intervention = 8 districts (73%)
Control = 7 districts (70%)
(high risk of bias)
Outcome group: Awareness – concern – interest
The response rate for teacher's awareness and interest was 69% (n = 432), and for concern was 52% (n = 324).
Insufficient reporting of attrition/exclusions to permit judgement of 'Low risk' or 'High risk' (e.g. no reasons for missing data provided) (unclear risk of bias).
Outcome group: Level of Use – District
District level grouping of Level of Use instruments sent to teachers (252/570 returned, response rate 44%). Mean percent of implementation for each district reported.
Intervention = 9 districts (82%)
Control = 7 districts (70%)
Imbalance across intervention and control district response (high risk of bias).
Outcome group: Level of Use – Teacher
Of the 570 Level of Use instruments sent to teachers, 252 were returned and completed, for a response rate of 44%. Only the Level of Use instruments for School Health and Tobacco Education Project curricula were analysed (n = 71).
Intervention = 52 teachers
Control = 20 teachers
Insufficient reporting of attrition/exclusions to permit judgement of 'Low risk' or 'High risk' (e.g. no reasons for missing data provided) (unclear risk of bias).
Outcome group: Implementation Checklist – District
Initial implementation
Intervention = 8 districts (73%)
Control = 6 districts (60%)
Later implementation
Intervention = 8 districts (73%)
Control = 6 districts (60%)
Imbalance across intervention and control district response (high risk of bias).
Outcome group: Implementation Checklist – Teachers
During the third year, 115 Implementation Checksheets were sent to teachers and 69 were returned (60%). Later implementation was measured in year 4 of the project. During this year, 136/175 (78%) Implementation Checksheets were returned.
Insufficient reporting of attrition/exclusions to permit judgement of 'Low risk' or 'High risk' (no reasons for missing data provided) (unclear risk of bias).
Selective reporting (reporting bias) Unclear risk No study protocol, therefore, it was unclear if there was selective outcome reporting.
Other bias Low risk Did not appear to be at risk of contamination as districts were the unit of randomisation.
Recruitment to cluster Unclear risk District selected which 2 middle or junior high schools participated.
Baseline imbalance Low risk Districts randomised to condition.
Loss of cluster Low risk Small loss of clusters (1 experimental, 1 control) and equal dropout across conditions.
Incorrect analysis High risk No adjustment for clustering reported in analysis.
Compatibility with individually randomised RCTs Unclear risk Unable to determine if a herd effect existed.
Overall risk of bias assessment Unclear risk Most domains were at unclear risk of bias.