Table 1.

Recommendations arising from the experiences of the SCORE secretariat and examples of these experiences

Recommendation	Benefits and considerations	Examples of SCORE experiences
Broad engagement
Engage the broad community —both those who are or will be funded and those who will not	Broad input helps ensure that the most relevant questions are asked. It supports dissemination and is more likely to impact practice and follow-up research.	Broad engagement took time and resources. The BMGF grant encouraged rapid roll-out of studies, requiring the SCORE secretariat to balance the time required to get broad input with the need to quickly implement sub-awards.
		Involvement of the WHO in planning meetings and as an ad hoc advisory committee member stimulated development of guidance related to point-of-care circulating cathodic antigen. Results from the large field studies are currently being used to inform new guidelines.
Partner with country NTD control programs in the research and to explore the implications of findings	Engaging NTD control programs ensures relevance of the studies and helps speed uptake of results. Because NTD control programs are critical to the WHO’s efforts, their involvement provides additional links to WHO recommendation and guideline development.	Engaging NTD control program managers who had not previously been involved with research in their countries required time and resources. In some places, SCORE researchers had difficulty engaging program managers or engagement was limited because of frequent program manager turnover.
Trade-offs and balances
Recognize the limitations imposed by resources and capacity, and design studies accordingly	An overly ambitious study is likely to fail.	The SCORE gaining and sustaining control studies were considered to be at the limit of what the research sites could handle. Some critical questions had to be excluded. All the studies were completed and provided useful data.
Balance rigor and relevance	The large field studies all used a cluster-randomized design—which is considered a strong research design. The main intervention, MDA, was implemented in the context of ongoing programs.	The randomization did not always result in balanced confounders among arms. Unexpectedly, high variability in village responses to MDA resulted in less power than anticipated. Integration with country programs took additional time and resources.
Weigh the risks and potential knowledge gained by conducting field studies in places that might have “red flags” on first consideration	Initially, SCORE and others were reluctant to conduct the major elimination study in Zanzibar. Some of the sites with little infrastructure or with instability (e.g., Mozambique) appeared to offer tremendous opportunities for the gaining and sustaining control studies.	The concern that work in Zanzibar would not be seen as generalizable appears now to be unfounded; the long history of MDA and political commitment helped make the study successful. In Mozambique, lack of school attendance and inadequate sensitization led to very low coverage levels. Nevertheless, the impact of MDA was clear. Political instability led to a delayed start of the sustaining control study in Côte d’Ivoire.
In multiyear studies, be visionary but cautious in incorporating new technologies	Researchers and staff appreciate the opportunity to be on the cutting edge of using new technologies. Significant resources can be saved if time allows delaying efforts to take advantage of rapidly developing technologies.	Had personal digital assistants been used for data collection in the gaining and sustaining studies, they would have been obsolete by the time the studies ended. The gaining and sustaining control studies were being implemented as smartphone data collection systems were developing. Keeping the systems simple would have made development and testing easier. For population genetics studies, it was possible to wait for technology to catch up to the study needs, resulting in more efficient processing of specimens, but creating a delay in obtaining results.
Management and oversight
Include adequate resources and personnel centrally for early and frequent on-site visits, especially for large field studies	Limiting secretariat visits and staff for close oversight allows for more resources for study implementation and possibly for more ownership of the studies by sites.	Greater oversight would have led to earlier identification of problems and higher quality implementation. Principal investigators and northern partner institutions were not always as attentive to the protocol and quality of study implementation as expected, and greater oversight might have addressed this issue.
Ensure resources are available for contingencies and opportunities	Resources are needed to address perturbations, for example, study delays due to political and other concerns, and to use the research infrastructure to address questions that arise in the course of the research.	Without the flexibility on the part of the BMGF and SCORE to address unexpected issues, some studies would not have been completed. Some critical SCORE results have come from studies added as opportunities or new questions were identified during the course of the core studies.
Intervention implementation and measurement
Ensure the primary intervention is adequately resourced and has appropriate oversight	SCORE attempted to walk a fine line between encouraging research-level implementation of interventions and implementation that might be realistic for programs.	In some of the gaining and sustaining control studies, poor coverage may have contributed to failure to see differences in outcomes among study arms. More resources and earlier implementation of snail control and behavioral interventions in the Zanzibar elimination study might have resulted in a greater impact in the arms involving these interventions.
Invest in measuring MDA quality and process measures related to MDA coverage	Better process measures related to the steps required for a successful MDA (e.g., what is being done for sensitization) and better coverage data might help identify problems earlier in the studies and provide information that could help explain results.	Collection of quality data requires significant resources. In places like Mozambique, with frequent large population shifts, an attempt to estimate denominators without investing in a repeat census was not successful. Coverage estimates by community drug distributors were shown to be overestimates in the Kenya gaining control study. Formal coverage survey evaluators should be considered for studies in which MDA is a primary intervention.
Measurement and analysis of outcomes
Define data elements in advance and set quality standards	Up-front investment in study design ensures higher quality and reduces time spent cleaning data and resolving data issues.	Initially, SCORE relied on sites with long histories of collecting data on similar outcomes, for example, infection status, to provide quality data. This did not work adequately, requiring significant amounts of time reconstructing and fixing issues related to data quality.
Develop SAPs before analysis of field study data	Explicitly defined analyses are standard for clinical trial research to avoid selective publication of only those results that support the study’s hypotheses. Analyses and publications using the SAP allows for easy comparisons of results from different studies using the same design.	Although the SCORE studies are somewhat structured like clinical trials, many aspects of the studies are not controlled. For the SCORE studies, some investigators questioned the time and effort involved in developing the SAPs. However, the SAPs ultimately were useful in structuring analysis and allowing for comparisons among studies.
Ensure that data instruments and collection for additional analyses are adequate for the intended purpose and provide training needed to collect quality data	The SCORE field studies provided opportunities to evaluate questions such as the cost of alternative approaches to MDA and whether village-level indicators could help in assessing force of transmission.	The data collection systems for cost and village-level indicators in the gaining and sustaining studies were used in most sites without adequate pretesting.

BMGF = Bill & Melinda Gates Foundation; MDA = mass drug administration; NTD = neglected tropical diseases; SAPs = standardized analysis plans; SCORE = Schistosomiasis Consortium for Operational Research and Evaluation; WHO = World Health Organization.