Feedback Comments	Author response
Comment: The research team at GiveWell has a handful of clarifying questions for the authors of the review on deworming treatments for children.
Was there a protocol for the most recent update to the Cochrane Review? If so, could it be shared?	Updates are broadly guided by the original protocol and review and standard practice is to document the changes made in the “history” section. This protocol was first published in 1997 and the first edition of the review was published in 1998. At this time there was no online repository for Cochrane protocols; we have therefore made the original protocol available via the "Related content" section here: http://cidg.cochrane.org/our‐reviews Cochrane policy is that when a team continue updating a review where the question and inclusion criteria stay the same, the team draw on new information, comments and criticisms, and a review of the current debates, background, objectives, inclusion criteria and methods (see Table 2. In Garner et al. BMJ 2016; 354: i3507). No fresh protocol is prepared unless it is a new team, or there are substantial changes to the inclusion criteria or methods used in the analysis. However, the author team should ensure the changes are transparent and summarized in the “What’s new/history section” and that is present.
‐The "History" section at the end of the review notes: "We changed the classification of Stephenson 1989 and Stephenson 1993. Previously these trials were in the ‘all children in an endemic area’ category, whereas now they are classified in the 'children with infection'. This decision was based on reviewing the trials with parasitologists and examining the prevalence and intensity of the infection where clearly the whole community was heavily infected” (p. 154). Could any information be shared about the process of consulting parasitologists on this topic or the output of those consultations?’	The Stephenson studies were reviewed as part of our last update, since they were a source of heterogeneity. We were examining how best to take this into account. We noted that in the methods section the authors noted: "The subjects consisted of all available children in the lower grades (Standards I and II) in Mvindeni Primary School in Kwale District, Coast Province, Kenya, an area where our previous work had shown that virtually all of the primary schoolchildren had hookworm (predominantly Necator americanus) and T. trichiura infections and that 50% were infected with A. lumbricoides."We had missed this information earlier. We consulted with LSTM parasitologists on this. They noted that virtually everyone was infected, and most were infected with at least two parasites and at least a third with three. In addition, the average hookworm loads put all the children into the moderate/heavy infection category. This is why this population was selected for the Stephenson studies. In this respect, the population chosen were equivalent to a population that had been screened to just include infected children. These indeed were quite old studies. So we made a decision that these studies were wrongly included in “treating the whole community” as everyone in the study population was infected. Hence they were reclassified.
‐Did you consult parasitologists about Watkins 1996? If so, how did you reach the conclusion to include that study in the “all children in an endemic area” category?	Our reading of the Stephenson studies was that the intention was to include a population where all children were infected. You ask about some other studies and why these were not reclassified as well (Watkins, Cruz, and Pollitt 1996). These were not in such high prevalence areas, but we take the point about the need to be systematic and will indeed have a closer look at their background prevalence in the update of the review.
‐Could you share any information about the rationale for the change in your classification schema from using "target population treated" to "all children in an endemic area" and "screened for infection" to "children with infection"? Does the change affect the classification of any studies included in the 2012 review other than Stephenson 1989 and Stephenson 1993?	This is because we judged that this was a better way of doing it. If all the children were infected (either because of the massively high worm infection burden, or as a result of screening), this was a clear way to describe the population. This is the whole reason for carrying out updates, to refine the analysis and make it clearer for the reader.
‐Is it the case that the Stephenson 1989 and Stephenson 1993 involved populations where every individual was infected? If not, was there a clear process for determining which studies fit under the "children with infection" classification? We are particularly curious about the rationale for including Stephenson 1993 under the "children with infection" classification while excluding Watkins 1996 from that classification.	As above. Virtually all of the children in the Stephenson studies had hookworm and Trichuris, and half had ascaris. Indeed, the intention of the authors was to select them on this basis.
‐Croke et al. reported that adding Stephenson 1993 back into a fixed effects version of Analysis 4.1 leads to a statistically significant weight effect, but they do not appear to report the random effects meta‐analysis result (Croke at al. 2016, Table 2, p. 27). How would adding the relevant Stephenson 1993 result affect the random effects meta‐analysis results in Analysis 4.1?	When we realised that Stephenson 1993 was in an area where everyone included was infected we moved the study into a more appropriate comparison, as outlined above. We stand by this analysis and change. What you are proposing is not a sensitivity analysis, but seems to be “what would we get if we did this‐and would it be significant?” We believe it is not helpful to shift the study around or tweak the statistical analysis retrospectively as there is a risk of the analysis being driven by the outcome of the analysis rather than first principles of whether the analysis is appropriate. In addition, statistical significance is not a critical flag of whether something works: the size of the effect is also critical (see below).
‐If Analysis 4.1 resulted in a statistically significant weight gain, would the authors still maintain their position that mass deworming of children in endemic areas “does not improve average nutritional status" (p. 2)?	It is not just a matter of statistical significance. There is a danger in chasing whether a result is statistically significant, this can be misleading, particularly when combined with multiple analyses of the same data. What is more important in drawing conclusions with limited and mixed data is to consider heterogeneity in the meta‐analysis, and to interpret the results in light of this. The GRADE approach is used in the review, and the assessment takes into account the effect size, the precision, the risk of bias, the directness of evidence, and heterogeneity between estimates. The GRADE assessment draws on the estimate of weight change from the main analysis (0.08 kg, 95%CI 0.11 to 0.27; analysis 4.1); and the GRADE uses a sensitivity analysis (6.1). In this analysis, which includes only studies with low risk of bias for allocation concealment, there was no evidence of an effect (0.01, 95%CI ‐0.13 to 0.15; analysis 6.1). This analysis is dominated by a single study, so to double check our inferences for this response, we conducted a further sensitivity analysis with studies at clear risk of bias excluded (Awasthi 2000, and Awasthi 1995); this provides an estimate of ‐0.01 kg (95% CI ‐0.15 to 0.13). Thus our published estimate and GRADE stand, downgraded on risk of bias and inconsistency, and we conclude “there may be little to no effect on weight" based on the main analysis estimate.