Country |
Study ID |
Study design |
Intervention independent of other changes |
Shape of the intervention effect prespecified |
Intervention unlikely to affect data collection |
Knowledge of the allocated interventions adequately prevented during the study |
Incomplete outcome data adequately addressed |
Selective outcome reporting |
Other risks of bias |
Brazil |
Viñuela 2015 |
ITS |
Unclear: other reforms were happening in the education and justice sectors that could have contributed as well. |
Low – specified as per guidance. |
Unclear: intervention may have affected data collection. |
Low: unlikely allocation affected data collection. |
Unclear: not specified. |
Low |
Note: data were aggregated at high level – this may have impacted analyses and findings. |
Cambodia |
Ir 2015 |
ITS |
High: multiple PBF reforms introduced alongside voucher schemes and changes to health service delivery (more trained professionals) also occurred. |
High – as per guidance, effect shape not specified. |
Unclear: intervention may have affected data collected as same source was used for payments and for outcome assessment. |
Unclear: health workers themselves appeared to be reporting. |
Unclear: not specified. |
Low |
Low |
Khim 2018a |
ITS |
Unclear: not specified. |
Low – specified as per guidance. |
Unclear: intervention may have affected data collection. |
Low: unlikely allocation affected data collection. |
Unclear: not specified. |
Low |
Note: several other schemes were implemented at the same time and high variability in implementation of this scheme noted. |
Matsuoka 2014 |
ITS |
Unclear: not specified. |
Low – specified as per guidance. |
Unclear: intervention may have affected data collection. |
Unclear: not specified. |
Unclear: not specified. |
Low |
Note: data reanalyzed. |
China |
Chang 2017 |
ITS |
High: other interventions concurrent (including further PBF and introduction of database). |
Low – specified as per guidance. |
High: intervention introduced alongside an HMIS intervention. |
Unclear: not specified. |
Unclear: not specified. |
Low |
Note: 3 PBF schemes implemented buy only 1 assessed. |
Wu 2014 |
ITS |
Unclear: other reforms happening but robustness checks performed to ascertain impacts and effects were consistent. |
Low – specified as per guidance. |
Low: no effects on data collection. |
Low: unlikely allocation affected data collection. |
Unclear: not specified. |
Low |
Note: not generalizable, study conducted in 1 setting. |
Liu 2005 |
ITS |
High: other changes in the country likely to affect trends. |
Low – specified as per guidance. |
Low: no effects on data collection. |
Low: blinded and random assessments. |
Low: panel dataset. |
Low |
Low |
Rwanda |
Rusa 2009a |
ITS |
High: other changes in the country (user fee removal) likely to affect trends. |
Low – specified as per guidance. |
Unclear: intervention may have affected data collection. |
Unclear: not specified. |
Unclear: not specified. |
Low |
Low |
Zambia |
Chansa 2015 |
ITS |
Unclear: not specified. |
Low – specified as per guidance. |
High: intervention introduced alongside audits. |
Low: unlikely allocation affected data collection. |
Low: HMIS data. |
Low |
Low |
Malawi |
McMahon 2016 |
CBA and ITS |
Unclear: not specified. |
Low – specified as per guidance. |
High: intervention directly targets improvements in data. |
Unclear: not specified. |
High: several indicators excluded due to missingness. |
Low |
Low |
CBA: controlled before‐after; HMIS: health management information system; ITS: interrupted time series; PBF: performance‐based funding. |