Glazerman 2012.

Methods	Stepped‐wedge design with 2 types of control groups (depending on the outcome measure): 1‐ Random assignment of schools to year of implementation 2‐ Propensity‐score matching of intervention schools to non‐intervention schools in the district
Participants	34 elementary schools in Chicago, USA Random assignment as follows: Cohort 1: 8 schools to start Chicago Teacher Advancement Program (TAP) in 2007 Cohort 2: 8 schools to start Chicago TAP in 2008 (serving as control for Cohort 1) Cohort 3: 9 schools to start Chicago TAP in 2009 Cohort 4: 9 schools to start Chicago TAP in 2010 (serving as control for Cohort 3) The control group was used for the outcome of teacher attitudes. Three hundred matched comparison schools were used as controls for the outcome of teacher retention. For teacher attitudes, a sample size of 270 to 278 teachers is reported, without any indication of the number of teachers in the intervention and control groups. For teacher retention, a sample size of 1102 teachers is reported for one‐year rates, 881 teachers for two‐year rates, and 781 teachers for three‐year rates. There is no indication of the specific number of teachers in the intervention and matched comparison schools.
Interventions	1‐ Chicago TAP‐ school‐wide intervention whereby teachers can earn extra pay and take on increased responsibilities through promotion, and become eligible for annual performance bonuses based on a combination of their contribution to student achievement and observed performance in the classroom. 2‐ No‐intervention (wait list) control
Outcomes	Teacher retention‐ measured as percentage of teachers returning to work at the same school in subsequent academic years at 12, 24, and 36 months post intervention Teacher attitudes (school climate) unvalidated survey, conducted at baseline and subsequently 12, 24, and 36 months post intervention
Notes	The theory of change for TAP is that giving teachers performance incentives, along with tools to track their performance and improve instruction, will help schools attract and retain talented teachers and help all teachers raise student achievement Date of the intervention: 2007‐2011 Funding: U.S. Department of Education Teacher Incentive Fund (TIF) granted to Chicago Public Schools (CPS) Conflict of interest: No
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Lottery conducted in May 2007 for Cohorts 1 & 2, and March 2009 for Cohorts 3 & 4 Stratified random sampling was utilised. Quote: "We used the many observable characteristics of schools to improve the chances that schools assigned to the treatment versus control group would have similar characteristics....In assigning cohorts 1 and 2 we imposed constraints on the randomisation so that the largest and smallest school (in terms of student enrolment) were in the same Chicago TAP cohort (treatment or control), the three schools with a predominantly non–African American student body were not in the same cohort, and neither cohort had more than one pair of schools from the same geographic area of the city. In assigning cohorts 3 and 4 we imposed constraints so that the difference in average enrolment between cohorts was less than 100 students, the difference in average total score on the Illinois Standards Achievement Test (ISAT) was less than six scale points, the three schools where fewer than 95 percent of students were African‐American were not all in the same cohort, and both cohorts had representation of each geographic area that contained more than one school".
Allocation concealment (selection bias)	Low risk	After the lottery assigning schools into cohorts, the research team sent the school assignments to the local education authority, who subsequently informed the school principals of their assignment
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not reported, not applicable
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not reported
Incomplete outcome data (attrition bias) All outcomes	Low risk	Attrition was accounted for and ITT analysis conducted
Selective reporting (reporting bias)	High risk	Retention is reported for Cohort 1 as 1‐year, 2‐year, and 3‐year rates. However, they are not reported independently for each of the other cohorts, and only reported in combination
Recruitment bias	Unclear risk	Not reported
Baseline imbalance	Low risk	Baseline characteristics have been reported for TAP schools (Cohorts 1 & 3), control schools (Cohorts 2 & 4), and matched‐comparison schools. No significant baseline differences in school characteristics, and in teacher age, gender, race, and experience (P > 0.05). The only significant difference (P < 0.05) was in the percentage of teachers who have a master's degree or higher between Cohort 3 TAP schools and the matched‐comparison group. However, there were no significant differences between other indicators of educational level such as 'National board certification' and 'Alternative education'.
Incorrect analysis	Low risk	Clustering of teachers within schools was accounted for, and impact estimates were calculated as regression‐adjusted means
Other bias	Unclear risk	There is no clear indication of the number of teachers in each cohort and in total