Skip to main content
PMC home page
HRB Open Research logoLink to HRB Open Research
. 2021 May 13;3:20. Originally published 2020 May 4. [Version 3] doi: 10.12688/hrbopenres.13041.3

Barriers and facilitators to the implementation of nutrition standards for school food: a mixed-methods systematic review protocol

Breda O'Mahony 1,2,a, Claire Kerins 2, Celine Murrin 3, Colette Kelly 4
PMCID: PMC7372527  PMID: 32743340

Version Changes

Revised. Amendments from Version 2

Following peer review, the authors have updated this manuscript to incorporate reviewer feedback. Specifically: -Moved some information from Step 2, part b (Coding of extracted data to the TDF) to Step 1 (Data extraction) -Grammatical updates

Abstract

Background: The importance of nutrition during childhood and the high prevalence of child and adolescent obesity has resulted in several countries implementing nutritional standards for school food as a way of providing healthy school food environments. Yet, there has been less focus on the barriers and facilitators influencing the process of implementing school food standards. This mixed-methods systematic review aims to address this evidence gap by synthesising the empirical evidence on the factors that may influence the implementation of school food standards.

Methods: This mixed-methods systematic review will use qualitative, quantitative and mixed-methods evidence from peer-reviewed publications retrieved from the following databases; PubMed, CINAHL, Scopus, EMBASE, Medline, PsycINFO and Web of Science. Grey literature will be accessed through Google Scholar, Open Access Theses and Dissertations, OpenGrey, RIAN, EThOS, ProQuest, WorldCat, Networked Digital Library of Theses and Dissertations, and public health organisation websites will also be accessed. Screening reference lists and citation chaining of all included studies will also be undertaken. No restrictions on publication date or language will be applied, however, only primary research studies relevant to supply-side stakeholders will be eligible for inclusion. Study quality will be assessed using the Mixed Methods Appraisal Tool. Study titles and abstracts will be screened to decide whether the full-text manuscript should be retrieved. For screening reliability, a second review author will assess a random sample of 20%. Kappa statistics will be used to assess inter-rater reliability, with values of 0.75 and higher representing high agreement. Two authors will independently extract data and factors reported to influence implementation. This will be synthesized using the Theoretical Domains Framework.

Discussion: A comprehensive understanding of these factors can provide guidance to relevant stakeholders to enhance the adoption, implementation and sustainability of nutrition standards for school meals.

Systematic review registration: PROSPERO CRD42019117904

Keywords: Mixed methods, Systematic review, Barriers, Facilitators, School meal standards, School food guidelines, Implementation, Nutrition

Abbreviations

MeSH, Medical Subject Headings; MMAT, Mixed Methods Appraisal Tool; PRISMA-P, Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols; PROSPERO, International Prospective Register of Systematic Reviews; TDF, Theoretical Doman Framework; NSLP, National School Lunch Programme; V2, Version 2.

Background

Schools are a key setting for the promotion of health and well-being 1– 3. They are one of the most effective ways of reaching a large segment of the population 4, 5, with no other institution having as much continuous contact and influence during the first stages of a child life 6.

One of the many ways that schools can support health is by the food that they provide 7. Good nutrition is associated with academic performance 8, psychological well-being and school attendance 9, 10. Up to a third of a child’s daily micronutrient intake can come from a school lunch 11, 12. Additionally, school meals can provide between 20 to 70% of a child’s energy requirements 13, thus further strengthening the need for healthy school meals. Coupled with this is a high prevalence of obesity among young people and the critical influence schools can play in supporting active living, healthy diets and body weight. This has resulted in many governmental school-based nutrition initiatives and policies, including nutrition standards for school meals being adopted 13. However, the effectiveness of school-based policies on childhood and adolescence nutrition and obesity depends on their implementation, which is often less than optimal, even when these policies are obligatory 14.

To date, a number of countries and regions around the world have introduced nutrition or food standards for school food on a mandatory basis. These include Finland in 1943 15, Sweden in 1997 16, Norway in 2001 17, a reintroduction of compulsory guidelines after 21 years in England in 2001 18, Slovenia in 2010 19, and an updated National School Lunch Programme in the United States of America (USA) in 2012, which will be phased into all schools by 2023 20. Nutrient standards are based on limits and promotions of various nutrients, whereas food-based standards set requirements on what food can and cannot be served and how frequently 21. Examples of standards include the Irish Nutrition Standards for School Meals 22. The food-based standards set out requirements for each meal type e.g. for breakfast, a minimum of two items should be provided, one serving of wholemeal or wholegrain cereals and breads, and one serving of either milk, yoghurt or cheese or fruit.

Differences exist in the provision of school food in Europe and internationally, and even from school to school within countries. Providing school food that meets nutritional guidelines or standards is complex 16, 23. Some countries provide school meals for all their students 24, regardless of their socio-economic environment 25, whilst in other jurisdictions the responsibility lies with the individual school 24. Other factors that contribute to the complex provision of school food include ensuring canteens make a profit 26 and organisational implications for principals 27. The latter includes existing contracts between food operators and schools that are agreed, based on the provision of catering infrastructure in schools 27.

It has been noted that more support must be provided to schools to allow the implementation of nutritional guidelines 28. Data from Gregoric and colleagues (2015) 13 found that the implementation of school nutrition guidelines differed by schools within the same region. Implementation as a process and a science is complex to study because of the numerous factors affecting implementation, its process, results and potential solutions 29.

Critical stakeholders involved in the implementation of food-based guidelines are supply-side stakeholders 23, 30 i.e. food service directors, catering managers and staff, school management, programme coordinators and contracted catering suppliers. Some of the factors related to implementation in schools include difficulties associated with preparing and serving fresh food at school 20; inadequate canteen facilities 31– 33; spending excessive time completing funding applications 30; and the requirement of staff training around the food guidelines 20, 34, 35. Positively, in contrast, caterers in the UK found the food standards relatively easy to achieve 36. However, there has been little synthesis of this research, particularly from the perspective of supply-side stakeholders.

Developing and improving strategies to increase supply-side stakeholders conformity of school meal standards requires a comprehensive understanding of the factors that enable and hinder implementation. One such framework that can allow us to apply theory to comprehensively identify factors that need to be addressed is the Theoretical Domains Framework (TDF) 37. The TDF was developed from 128 theoretical constructs from 33 theories that were perceived to be most relevant to implementation questions 38, 39. It was first published in 2005 40 but later validated in 2012 (version 2 (v2)) 41. It has been used in numerous reviews to understand barriers and facilitators to a wide variety of behaviours 41– 43 and has confirmed validity and reliability 44, 45. Such reviews include implementation of dietary guidelines in early childhood education centres in Australia 46 and barriers and facilitators to the implementation of physical activity policies in schools 47. The framework (v2) provides 14 domains, which can capture a range of factors that influence implementation outcomes. These include knowledge, skills, memory, attention and decision processes, behavioural regulation, social/professional role and identity, beliefs about capabilities, optimism, beliefs about consequences, intentions, goals, reinforcement, emotion, environmental context and resources, and social influences 48.

A number of studies have identified various factors that influence implementation. Yet there is no published systematic review in this area that has adopted a bottom-up policy implementation perspective by focusing on the experiences and views of supply-side stakeholders, a key group in the adoption of these standards. Given this evidence gap, the primary aim of the systematic review is to collate the factors that influence the implementation of nutrition or food standards for school food provision in primary and post-primary settings (children aged 5–18 years). The use of the TDF to synthesise findings provides a mechanism to identify theoretical constructs to target in the development of food standards/guideline implementation. Using a theory provides a strong foundation for policy development, in contrast to simply identifying the barriers and facilitators 49. Understanding these factors from a theoretical perspective will provide a list of modifiable factors to target. This will help to inform future planning, improve uptake and practice of standards. Essentially, this review can guide policy-makers, researchers and individuals responsible for devising and implementing nutrition standards in schools.

Review objectives

The primary objective is to identify and synthesise the existing evidence on the barriers and facilitators to implementing food or nutrition standards for school food from supply-side stakeholders. A secondary objective is to compare the barriers and facilitators between a primary and post-primary school setting.

Methods

This mixed-methods systematic review is registered with the international database of prospective systematic review; Prospective Register of Systematic Reviews (PROSPERO): CRD42019117904 (25 th June 2019). The Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) checklist has been assessed in the preparation of this protocol (see Reporting guidelines) 50. The review will be conducted in accordance with PRISMA statement guidelines.

Study eligibility criteria

The PICOS acronym (Population, Intervention, Comparison, Outcome and Study design) will be used to select study criteria, as described below. PICOS was selected due to achieving a comprehensive search with greater sensitivity than specificity 51.

Population. To be eligible for this review, studies have to include data which focuses on stakeholders who have a role in the implementation of nutrition or food standards or guidelines for school food within primary and post-primary school settings. Supply-side stakeholders refers but is not limited to catering management and staff, school principals/managers, contracted catering suppliers, food service directors and managers, and programme coordinators. It will also include studies that allude to officials from government organisations that may influence food provision in schools e.g. policy-makers. As this is an international review and to avoid differences that exist from country to country, e.g. age, all types of primary and post-primary schools will be included (Junior, Elementary, Middle, Secondary, Senior and High school). Standards in pre-schools, post-secondary schools and third-level settings will not be included. Furthermore, studies involving school children's perceived barriers and facilitators will also be excluded. Students are not included as they are classified as demand-side stakeholders.

Intervention. We will include studies of interventions delivered in educational establishments where the standards for school food have been implemented on a voluntary or mandatory basis. This includes food and nutrient standards for all meals, beverages and snacks provided in schools, including breakfast clubs and vending machines. As different jurisdictions implement different types of standards, both nutrient and food-based standards will be included. Studies on school nutrition policies and healthy eating interventions will not be included unless such policies and interventions are based on school meal standards. Similarly, studies on health promoting schools will not be included unless data specific to school-based standards can be extracted. The decision to include studies where voluntary school food standards were implemented is based on a preliminary scoping search.

Control. Whilst no comparator is being studied in this review, studies will not be excluded on the basis of having a comparator or control group.

Outcome. The primary outcome will include any barrier or facilitator to the implementation of nutrition and food-based standards for school food. For this review, we will use a similar definition that Kerins et al. applied in a systematic review protocol 52. A barrier is defined as any variable that impedes or obstructs the implementation of nutrition standards, whereas a facilitator is defined as any variable that eases and promotes the implementation of nutrition standards. The findings may include the following: (i) verbatim quotations from research participants; (ii) excerpts, quotations or entire passages from studies using documentary analysis; (iii) narrative descriptive summaries of results; and (iv) statistical analyses from surveys and questionnaires. A secondary outcome of the review is to compare the barriers and facilitators between a primary and post-primary school setting.

Study design. We are conducting a mixed-method systematic review; therefore quantitative, qualitative and mixed-method studies will be accessed. The rationale for this choice is to capture a comprehensive understanding of the factors that affect implementation. This may include, but is not limited to, the following studies which use appropriate methods of data collection and analysis (i) qualitative studies; case studies, grounded theory, ethnography, action research studies (ii) quantitative studies; case-control studies, quasi-experimental studies, randomised controlled trials, cross-sectional studies and (iii) mixed-methods (combining qualitative and quantitative methods of data collection and analysis); focus groups, interviews, surveys, questionnaires, observation. This review will disregard editorials, commentary and opinion pieces.

Language. There will be no restriction on language. This is to ensure all suitable international research on nutrient or food-based standards is captured.

Publication year. There will be no restriction on publication year.

Search strategy

A search of peer-reviewed literature combining, where possible, published search filters for school meals, barriers or facilitators, will be undertaken. Guidance of an experienced librarian and discussion amongst the review team will also take place to inform the strategy. Broad search terms will be used to garner greater sensitivity than specificity so as to ensure a comprehensive search is undertaken 53. Databases relating to various fields, including education, food, and nutrition will be used. Each search strategy will be database specific and will include applicable elements such as Medical Subject Headings (MeSH) (or equivalent), truncation, Boolean operators and will be adapted where appropriate. Initial scoping searches will be undertaken by the lead review author to refine the search strategy. Table 1 illustrates a sample search strategy for the CINAHL database. The following electronic databases will be searched: PubMed, CINAHL, Scopus, EMBASE, Medline, PsycINFO and Web of Science. To identify published government reports and other grey literature, searches through Google Scholar, Open Access Theses and Dissertations, OpenGrey, RIAN, EThOS, ProQuest, WorldCat, Networked Digital Library of Theses and Dissertations, and public health organisation websites will also be undertaken. Furthermore, this minimises the influence of publication bias and produces a balanced picture of available evidence 54. To identify any additional studies, the reference lists of all included studies will be screened to retrieve additional eligible articles 55. All search results will be reviewed for eligibility, except in the case of Google Scholar where the first 200 citations will be screened. A priori decision to screen the first 200 hits on Google Scholar, as sorted by relevance, was decided after considering the time required to screen each hit 53. The lead or corresponding authors for all included studies will be contacted (via email with two attempts) so as to identify on-going or unpublished research studies that may be relevant to this review. To ensure that the search strategy is undertaken in a systematic way, a memoing method will be used to record the working notes when conducting preliminary searches as well as documenting the protocol-driven search strategy 56.

Table 1. Sample CINAHL title and abstract search strategy.

Search number Search string
#1 recommend* OR guideline adherence [mh] OR guidance* OR protocol* OR nutrition
policy [mh] OR strateg* OR standard* OR nutri* OR health promotion [mh]
#2 school lunch* OR school meal* OR canteen* OR food services [mh] OR school food OR
menu planning [mh] OR food program* OR school* meal* program* OR school dinner*
#3 school* OR education*
#4 #1 AND #2 AND #3
Open in a new tab

Mh MeSH headings

Study selection

Data management. EndNote X9 will be used to manage references throughout the review. Once the searches have been carried out, the search results will be exported to EndNote. This will identify any duplicates, which will then be removed.

Screening. Search results will be imported into an online systematic review software, Rayyan. This will enable screening, data extraction and quality assessment. This will be undertaken after a piloted, clear and detailed set of inclusion and exclusion criteria has been drawn up (see Extended data) 50. The lead author will screen study titles and abstracts to decide whether the full-text manuscript should be retrieved. For screening reliability, a second review author will assess a random sample of 20%. Kappa statistics will be used to assess inter-rater reliability, with values of 0.75 and higher representing high agreement 57. In the case of disagreement, full article will be screened. Each study will be categorised into (a) potentially meeting the eligibility criteria or (b) not meeting the eligibility criteria. For all potentially eligible studies, full-text manuscripts will be obtained. A full-text screening process will then commence by two independent reviewers, which will then produce a final set of papers to be included in the review. In situations where the study eligibility cannot be resolved via consensus, a third review author will be consulted. A flow diagram will be completed to record the numbers of papers through each stage of the search and screening process, as recommended by the PRISMA guidelines 58.

Quality assessment/risk of bias

Quality appraisal will be conducted by two independent reviewers, using the Mixed Methods Appraisal Tool (MMAT) (2018 version). This assessment tool was selected as it is used to efficiently appraise the most common methods and methodologies i.e. qualitative, quantitative and mixed-methods studies, with few generic quality criteria 59, 60. Additionally, the tool was designed to appraise the methodological quality of studies in a mixed-methods systematic review and not the quality of report writing 61. The MMAT focuses on methodological criteria 59 and includes two screening questions and nineteen questions corresponding 61 to the following five categories of study design; qualitative research, randomised controlled trials, non-randomised studies, quantitative, observational descriptive, and for mixed-methods studies 59, 62. For each study type, reviewers will quality score using a MMAT table. When disagreements between reviewers cannot be easily resolved, a third independent reviewer will become involved in order to reach a consensus.

Data extraction and synthesis

For data extraction and synthesis, we have adopted a similar approach undertaken in a study protocol Graham-Rowe and colleagues 63. Data will be extracted from all full-text studies that meet the inclusion criteria, using a data extraction form. The data extraction form will be created using Microsoft Word. Data to be extracted will include, but is not limited to, the following: Key study information which will include study title, name of the first author, year of publication, country of study, language, study type (qualitative, quantitative and mixed methods studies), study design (e.g. cross-sectional and observational), intervention type (e.g. food standards, nutrient standards), type of implementation (voluntary or mandatory participation), educational setting (primary, post-primary, academies etc.) and participant characteristics (canteen staff, head-teacher, contracted catering suppliers, food service directors and managers, programme coordinators etc.), sample size, data collection and analysis methods. Data on intervention effects/outcomes, such as change in children’s dietary habits will not form part of this review. A coding manual with the definitions outlined by Cane et al., for the 14 theoretical domains from the TDF, will also be prepared.

Analysis

Step 1: Data extraction

Two review authors, not blind to author or journal information will independently extract from all the full-text studies that fulfil the inclusion criteria, using a data extraction form. Each data point will be categorised as either (a) raw data (e.g. quotes from qualitative studies); (b) analysed data from results sections (e.g. statistical analysis from surveys and questionnaires (c) interpretative descriptions and summaries of results from published reports. For qualitative studies, both the theme and the original reporting format of the barrier or facilitator will be extracted. This will include participant quotes, excerpts or entire passages from studies using documentary analysis, and narrative descriptive summaries of results. Whereas in quantitative studies, the frequency of all the barriers and facilitators will be extracted. Where mixed-method studies are included, data will be extracted into the relevant qualitative and quantative extraction forms.

Deductive analysis

Step 2: Coding of extracted data

   (a) Pilot of the coding of data

To ensure validity and reliability, a pilot be will undertaken to allow the review authors to practice coding extracted data into the TDF domains. This will enable comparison of data coding and implement modification, if required. This will be jointly undertaken by the two review authors using the domain definition manual. Any discrepancies will be resolved by the review team, and if necessary by a third reviewer.

   (b) Coding extracted data to the TDF

The lead author will continue to code the extracted data, using the domain definition manual, to the TDF domain that they judge to best represent the factor. If a reported barrier or facilitator is judged to represent more than one domain, it will be coded multiple times into its associated domains. Where a factor is not recorded as a barrier or facilitator, we will revert back to the definitions. This is important so as to be able to highlight the pertinent factors affecting implementation and provide relevant recommendations. To access coding reliability, a second review author will assess a random sample of 20%. Kappa statistics will be used again to assess inter-rater reliability, with values of 0.75 and higher representing high agreement.

Inductive analysis

Step 3: Thematic synthesis

Inductive thematic synthesis, based on methods by Thomas & Harden 64 will also be used to code any data that does not fit into the TDF. This will require a three-step process. Step one involves line by line coding of the data that was not coded to the TDF. The second step involves organisation or grouping of these code into associated areas to construct descriptive themes. In step three, the descriptive themes will be compared to refine the relationship between them so as to generate analytical themes.

Sensitivity analysis

Following the synthesis stage, the first author will perform a sensitivity analysis. This will determine if the review findings are sensitive to the following variables: study quality, study methodology (qualitative, quantitative or mixed methods), type of implementation (voluntary or mandatory participation) and location. Both bias and sensitivity reviews will help ensure quality, rigour and transparency 65

Importance criterion

To determine which domains are likely to be the most important factors influencing implementation, each domain in step two will be reviewed. In this review, the frequency of each domain will be examined. Similar to a study protocol by Graham-Rose et al. 63, for qualitative studies we will then consider ‘expressed importance’ within each domain by looking for a statement(s) from the authors interpretation of the study findings articulating what beliefs were reported to be important by the study participants. We acknowledge that this is not a precise process but it has a good fit with the qualitative approach as the meaning, interpretation and prioritisation of the data will be by the authors who have closer familiarity with the primary data, than will be possible by the review team. This process will require discussion and critical judgement by the research team. This will allow the review team, to interpret the domains that have the highest frequency and ‘expressed importance’ as the most important factors in the implementation of school meals standards/guidelines.

If there is sufficient data, we will explore whether the domains identified as important vary according to the educational establishment i.e. primary and post-primary schools. This will be performed by subgroup analysis. Findings will be described in tabular format.

Study status

This study has not yet commenced.

Discussion

The internal school food environment is considered to have a significant influence on student’s food consumption 66. It is believed that over 35% of their energy is obtained at school 67. Moreover, in many instances a school meal may be the only complete meal that students have access to 68. In response to the need of schools to play a more supportive role in obesity prevention 69, many jurisdictions have implemented policies and practices, one of which is food or nutrition standards for school meals 13, 70.

It is believed that the UK has the most comprehensive set of nutritional standards for school meals. However, the implementation of these standards has not necessarily resulted in better consumption and nutritional outcomes 71. Therefore, it is important to evaluate the process, to aid the full implementation of nutrition standards. Implementation evaluation measures the results from a process 72 and enables the transformation of policy plans into action 73. However, there are many individual, environmental and socio-cultural factors that can affect the successful implementation of policies 49. This is particularly pertinent to schools which are complex, with numerous factors that can influence implementation 74, the quality of implementation and the expected outcomes of the policy 75. The use of the TDF provides a holistic approach as it considers the complex interaction of the how and why 76, which must be taken into account when considering how nutrition standards for school meals are implemented in school settings.

A number of studies have identified various factors that affect implementation, however, there has been limited synthesis of this information from a supply-side stakeholder’s perspective. Ronto and colleagues (2020) 77 recent systematic review analysed the implementation and/or compliance with school-based healthy food and beverage policies. The eligibility criteria included policy relating to nutrition guidelines, regulations and/or restrictions on food and beverage availability, advertisement, placement or price. There was no exclusion criteria in relation to study participants, and grey literature was not accessed. Our mixed-methods review is specific to the implementation of food/nutrient-based standards from a supply-side stakeholder’s perspective using a variety of electronic databases and grey literature sources. Given the potential impact that school meal guidelines and standards can have on the health and well-being outcomes of children and adolescents, understanding the factors that affect their implementation is key. We are confident that the depth of this review will provide a holistic understanding of the factors as all types of studies, qualitative and quantitative or both, including grey literature, will be accessed. Furthermore, there will be no language or publication date restrictions. The review will follow academic rigour and will include a number of strategies for validity, reliability and to reduce the effects of bias. This will be achieved by having clear and detailed inclusion and exclusion criteria, independent reviewers, the use of PRISMA guidelines, a MMAT, and by using computer packages for data and quality management. Finally, where deviations from this protocol occur, this will be justified and discussed in the systematic review upon publication and will be documented on PROSPERO.

The outcomes of this study will be applicable to policy-makers and their advisors, practitioners, researchers and school administrators responsible for supporting the implementation of nutrition standards. Documenting barriers is necessary to improve the implementation of policy changes 49. Furthermore, a theoretical based framework will be used, which will provide a greater insight into the complexities of implementation. It will also have the capacity to steer future developments and implementations.

When completed, the review results will be submitted for publication to a peer-reviewed journal with the potential of writing a policy brief targeted at key stakeholders. Where applicable and accepted, findings will be disseminated and communicated at conferences, workshops, seminars, and via social media.

Data availability

Underlying data

No underlying data are associated with this article.

Extended data

Open Science Framework: Barriers and facilitators to the implementation of nutrition standards for school food: a mixed-methods systematic review, https://doi.org/10.17605/OSF.IO/6Q24P 50.

This project contains the following extended data:

  • -

    Supplementary File 2. Inclusion and Exclusion Criteria.pdf

Reporting guidelines

Open Science Framework: PRISMA-P checklist for ‘Barriers and facilitators to the implementation of nutrition standards for school food: a mixed-methods systematic review’, https://doi.org/10.17605/OSF.IO/6Q24P 50.

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CCO 1.0 Public domain dedication).

Funding Statement

Health Research Board [SPHeRE/2013/1]. This research is funded by the Further Education Policy (FEP) Scheme at National University of Ireland, Galway (NUIG).

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[version 3; peer review: 2 approved, 1 approved with reservations]

References

  • 1. Nelson M, Breda J: School Food Research: Building the Evidence Base for Policy. Public Health Nutr. 2013;16(6):958–967. 10.1017/S1368980012005162 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. European Union: EU Action Plan on Childhood Obesity 2014–2020. European Union;2014. Reference Source [Google Scholar]
  • 3. World Health Organisation: Adolescent’s Dietary Habits. World Health Organisation;2016. Reference Source [Google Scholar]
  • 4. Story M, Kaphingst KM, French S: The role of schools in obesity prevention. Future Child. 2006;16(1):109–142. 10.1353/foc.2006.0007 [DOI] [PubMed] [Google Scholar]
  • 5. Wechsler H, Devereaux RS, Davis M, et al. : Using the school environment to promote physical activity and healthy eating. Prev Med. 2000;31(2):S121–S137. 10.1006/pmed.2000.0649 [DOI] [Google Scholar]
  • 6. Story M, Nanney M, Schwartz M: Schools and Obesity Prevention: Creating School Environments and Policies to Promote Healthy Eating and Physical Activity. Milbank Q. 2009;87(1):71–100. 10.1111/j.1468-0009.2009.00548.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Frerichs L, Brittin J, Sorensen D, et al. : Influence of School Architecture and Design on Healthy Eating: A Review of the Evidence. Am J Public Health. 2015;105(4):e46–57. 10.2105/AJPH.2014.302453 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Belot M, James J: Healthy School Meals and Educational Outcomes. J Health Econ. 2011;30(3):489–504. 10.1016/j.jhealeco.2011.02.003 [DOI] [PubMed] [Google Scholar]
  • 9. Jacka F, Kremer P, Berk M, et al. : A Prospective Study of Diet Quality and Mental Health in Adolescents. PLoS One. 2011;6(9):e24805. 10.1371/journal.pone.0024805 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Masse L, de Niet J: School Nutritional Capacity, Resources and Practices Are Associated With Availability of Food/Beverage Items in Schools. Int J of Behav Nutr Phys Act. 2013;10:26. 10.1186/1479-5868-10-26 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Nelson M, Lowes K, Hwang V: The contribution of school meals to food consumption and nutrient intakes of young people aged 4–18 years in England. Public Health Nutr. 2006;10(7):652–662. 10.1017/S1368980007382529 [DOI] [PubMed] [Google Scholar]
  • 12. Osowski CP, Lindroos AK, Barbieri HE, et al. : The contribution of school meals to energy and nutrient intake of Swedish children in relation to dietary guidelines. Food Nutr Res. 2015;59:27563. 10.3402/fnr.v59.27563 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Gregoric M, Pograjc LM, Pavlovec A, et al. : School nutrition guidelines: overview of the implementation and evaluation. Public Health Nutr. 2015;18(9):1582–1592. 10.1017/S1368980014003310 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Morris J, Zidenburg-Cherr S: Garden enhanced nutrition curriculum improves fourth grade children’s knowledge of nutrition and preferences for some vegetables. J Am Diet Assoc. 2002;102(1):91–3. 10.1016/s0002-8223(02)90027-1 [DOI] [PubMed] [Google Scholar]
  • 15. Pellikka K, Manninen M, Taivalmaa SL: School Meals for All. School feeding: investment in effective learning-Case Finland.2019. Reference Source [Google Scholar]
  • 16. Swedish National Food Agency: Good School Meals. Livsmedelsverket: Uppsala, Sweden,2014. [Google Scholar]
  • 17. The Norwegian Directorate for Health and Social Affairs: The National Guidelines for School Meals. (In Norweigian).The Norwegian Directorate for Health and Social Affairs: Oslo,2005. [Google Scholar]
  • 18. Adamson A, Spence S, Reed L, et al. : School food standards in the UK: implementation and evaluation. Public Health Nutr. 2013;16(6):968–81. 10.1017/S1368980013000621 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Uradni list RS: Zakon o šolski prehrani (ZŠolPre-1).Ljubljana,2008. Reference Source [Google Scholar]
  • 20. Tabak RG, Moreland-Russell S: Food Service Perspectives on National School Lunch Program Implementation. Health Behav and Policy Rev. 2015;2(5):362–371. 10.14485/HBPR.2.5.4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Haroun D, Wood L, Harper C, et al. : Nutrient-based standards for school lunches complement food-based standards and improve pupils' nutrient intake profile. Br J Nutr. 2011;106(4):472–4. 10.1017/S0007114511002297 [DOI] [PubMed] [Google Scholar]
  • 22. Department of Health: Nutrition Standards from School Meals.2017. Reference Source [Google Scholar]
  • 23. Asada Y, Ziemann M, Zatz L, et al. : Successes and Challenges in School Meal Reform: Qualitative Insights From Food Service Directors. J Sch Health. 2017;87(8):608–615. 10.1111/josh.12534 [DOI] [PubMed] [Google Scholar]
  • 24. World Health Organization: Food and Nutrition Policy for Schools. Copenhagen: WHO Regional Office for Europe.2006. Reference Source [Google Scholar]
  • 25. Lucas PJ, Patterson E, Sacks G, et al. : Preschool and School Meal Policies: An Overview of What We Know About Regulation, Implementation, and Impact on Diet in the UK, Sweden, and Australia. Nutrients. 2017;9(7): pii:E736. 10.3390/nu9070736 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Cohen JF, Gorski MT, Hoffman JA, et al. : Healthier Standards for School Meals and Snacks: Impact on School Food Revenues and Lunch Participation Rates. Am J Prev Med. 2016;51(4):485–92. 10.1016/j.amepre.2016.02.031 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Browne S, Barron C, Staines A, et al. : ‘We know what we should eat but we don’t …’: a qualitative study in lrish secondary schools. Health Promot Int. 2019; pii:daz087. 10.1093/heapro/daz087 [DOI] [PubMed] [Google Scholar]
  • 28. Abery E, Drummond C: Implementation of Mandatory Nutritional Guidelines in South Australian Primary School Canteens: A Qualitative Study. Improving Schools. 2014;17(1):41–53. 10.1177/1365480213519183 [DOI] [Google Scholar]
  • 29. Peters DH, Adam T, Alonge O, et al. : Implementation research: what it is and how to do it. BMJ. 2013;347:f6753. 10.1136/bmj.f6753 [DOI] [PubMed] [Google Scholar]
  • 30. Day RE, Sahota P, Christian MS, et al. : A qualitative study exploring pupil and school staff perceptions of school meal provision in England. Br J Nutr. 2015;114(9):1504–14. 10.1017/S0007114515002834 [DOI] [PubMed] [Google Scholar]
  • 31. Drummond C, Sheppard L: Examining primary and secondary school canteens and their place within the school system: a South Australian study. Health Educ Res. 2011;26(4):739–49. 10.1093/her/cyr036 [DOI] [PubMed] [Google Scholar]
  • 32. Valaitis RF, Hanning RM, Herrmann IS: Programme coordinators’ perceptions of strengths, weaknesses, opportunities and threats associated with school nutrition programmes. Public Health Nutr. 2014;17(6):1245–54. 10.1017/S136898001300150X [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Holthe A, Larsen T, Samdal O: The role of physical structures in implementing the Norwegian guidelines for healthy school meals. Health Place. 2010;16(1):93–100. 10.1016/j.healthplace.2009.09.001 [DOI] [PubMed] [Google Scholar]
  • 34. Weir CJ: Caterers’ experiences and perceptions of implementing the 2006 school meal standards. J Hum Nutr Diet. 2008;21(5):526. 10.1111/j.1365-277X.2008.00911_3.x [DOI] [Google Scholar]
  • 35. Yon BA, Amin SA, Taylor JC, et al. : School Nutrition Directors' Perspectives on Preparing for and Implementing USDA's New School Meal Regulations. J Child Nutr Manag. 2016;40(1). Reference Source [Google Scholar]
  • 36. Gray J: Implementation of school food standards in England - a catering perspective. Nutr Bull. 2008;33(3):240–244. 10.1111/j.1467-3010.2008.00712.x [DOI] [Google Scholar]
  • 37. Hall AM, Scurrey SR, Pike AE, et al. : Physician-reported barriers to using evidence-based recommendations for low back pain in clinical practice: a systematic review and synthesis of qualitative studies using the Theoretical Domains Framework. Implement Sci. 2019;14(1):49. 10.1186/s13012-019-0884-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Richardson M, Khouja CL, Sutcliffe K, et al. : Using the theoretical domains framework and the behavioural change wheel in an overarching synthesis of systematic reviews. BMJ Open. 2019;9(6):e024950. 10.1136/bmjopen-2018-024950 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Cane J, O'Connor D, Michie S: Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37. 10.1186/1748-5908-7-37 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Michie S, Johnston M, Abraham C, et al. : Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33. 10.1136/qshc.2004.011155 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. Atkins L, Francis J, Islam R, et al. : A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implement Sci. 2017;12(1):77. 10.1186/s13012-017-0605-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. French SD, Green SE, O'Connor DA: Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7:38. 10.1186/1748-5908-7-38 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Tuti T, Nzinga J, Njoroge M, et al. : A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implem Sci. 2017;12(1):61. 10.1186/s13012-017-0590-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Sarmast H, Mosavianpour M, Collet JP, et al. : TDF (Theoretical Domain Framework): how inclusive are TDF domains and constructs compared to other tools for assessing barriers to change? BMC Health Serv Res. 2014;14(Suppl 2):P81. 10.1186/1472-6963-14-S2-P81 [DOI] [Google Scholar]
  • 45. Huijg JM, Gebhardt WA, Crone MR, et al. : Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research. Implement Sci. 2014;9:11. 10.1186/1748-5908-9-11 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Grady A, Seward K, Finch M, et al. : Barriers and Enablers to Implementation of Dietary Guidelines in Early Childhood Education Centers in Australia: Application of the Theoretical Domains Framework. J Nutr Educ Behav. 2018;50(3):229–237.e1. 10.1016/j.jneb.2017.09.023 [DOI] [PubMed] [Google Scholar]
  • 47. Nathan N, Elton B, Babic M, et al. : Barriers and facilitators to the implementation of physical activity policies in schools: A Systematic Review. Prev Med. 2018;107:45–53. 10.1016/j.ypmed.2017.11.012 [DOI] [PubMed] [Google Scholar]
  • 48. Francis JJ, O’Connor D, Curran J: Theories of behaviour change synthesised into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci. 2012;7:35. 10.1186/1748-5908-7-35 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Weatherson KA, Gainforth HL, Jung ME: A theoretical analysis of the barriers and facilitators to the implementation of school- based physical activity policies in Canada: a mixed methods scoping review. Implement Sci. 2017;12(1):41. 10.1186/s13012-017-0570-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. O’Mahony B, Kerins C, Murrin C, et al. : Barriers and facilitators to the implementation of nutrition standards for school food: a mixed methods systematic review protocol.2020. 10.17605/OSF.IO/6Q24P [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51. Methley AM, Campbell S, Chew-Graham C, et al. : PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579. 10.1186/s12913-014-0579-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52. Kerins C, McSharry J, Hayes C, et al. : Barriers and facilitators to implementation of menu labelling interventions to support healthy food choices: a mixed methods systematic review protocol. Syst Rev. 2018;7(1):88. 10.1186/s13643-018-0752-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Bramer WM, de Jonge GB, Rethlefsen ML, et al. : A systematic approach to searching: an efficient and complete method to develop literature searches. J Med Libr Assoc. 2018;106(4):531–541. 10.5195/jmla.2018.283 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. Paez A: Grey literature: An important resource in systematic reviews. J Evid Based Med. 2017. [published online ahead of print, 2017 Dec 21]. 10.1111/jebm.12265 [DOI] [PubMed] [Google Scholar]
  • 55. Horsley T, Dingwall O, Sampson M: Checking reference lists to find additional studies for systemactic review. Cochrane Database Syst Rev. 2011;10(8):MR000026. 10.1002/14651858.MR000026.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Booth A: Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev. 2016;5:74. 10.1186/s13643-016-0249-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. Landis JR, Koch GG: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977;33(2):363–74. 10.2307/2529786 [DOI] [PubMed] [Google Scholar]
  • 58. Moher D, Liberati A, Tetzlaff J, et al. : Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59. Hong QN, FÀBregues S, Bartlett G, et al. : The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Education for Information .2018;34(10):1–7. 10.3233/EFI-180221 [DOI] [Google Scholar]
  • 60. Pluye P, Gagnon MP, Griffiths G, et al. : A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in Mixed Studies Reviews. Int J of Nurs Stud. 2009;46(4):529–546. 10.1016/j.ijnurstu.2009.01.009 [DOI] [PubMed] [Google Scholar]
  • 61. Souto RQ, Khanassov V, Hong QN, et al. : Systematic mixed studies reviews: updating results on the reliability and efficiency of the Mixed Methods Appraisal Tool. Int J of Nurs Stud. 2015;52(1):500–01. 10.1016/j.ijnurstu.2014.08.010 [DOI] [PubMed] [Google Scholar]
  • 62. Pace R, Pluye P, Bartlett G, et al. : Testing the Reliability and Efficiency of the Pilot Mixed Methods Appraisal Tool (MMAT) for Systematic Mixed Studies Review. Int J of Nurs Stud. 2012;49(1):47–53. 10.1016/j.ijnurstu.2011.07.002 [DOI] [PubMed] [Google Scholar]
  • 63. Graham-Rowe E, Lorencatto F, Lawrenson JG, et al. : Barriers and enablers to diabetic retinopathy screening attendance: Protocol for a systematic review. Syst Rev. 2016;5(1):134. 10.1186/s13643-016-0309-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64. Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45. 10.1186/1471-2288-8-45 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65. Langer L, Stewart R, Winters N: Sensitivity analysis in systematic reviews of qualitative evidence: an example from a mixed-methods systematic review.In: Filtering the information overload for better decisions. Abstracts of the 23rd Cochrane Colloquium; 2015 3 7 Oct; Vienna, Austria. John Wiley & Sons;2015. Reference Source [Google Scholar]
  • 66. Kelly C, Callaghan M, Molcho M, et al. : Food environments in and around post-primary schools in Ireland: Associations with youth dietary habits. Appetite. 2019;132:182–89. 10.1016/j.appet.2018.08.021 [DOI] [PubMed] [Google Scholar]
  • 67. Briefel RR, Wilson A, Gleason PM: Consumption of low-nutrient, energy-dense foods and beverages at school, home, and other locations among school lunch participants and nonparticipants. J Am Diet Assoc. 2009;109(2 Suppl):S79–90. 10.1016/j.jada.2008.10.064 [DOI] [PubMed] [Google Scholar]
  • 68. Bevans KB, Sanchez B, Teneralli R, et al. : Children's eating behavior: the importance of nutrition standards for foods in schools. J Sch Health. 2011;81(7):424–9. 10.1111/j.1746-1561.2011.00611.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69. Swinburn B, Vandevijvere S: WHO report on ending childhood obesity echoes earlier recommendations. Public Health Nutr. 2016;19(1):1–2. 10.1017/S1368980015003663 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70. Welker E, Lott M, Story M: The School Food Environment and Obesity Prevention: Progress Over the Last Decade. Curr Obes Rep. 2016;5(2):145–55. 10.1007/s13679-016-0204-0 [DOI] [PubMed] [Google Scholar]
  • 71. Addis S, Murphy S: 'There is such a thing as too healthy!' The impact of minimum nutritional guidelines on school food practices in secondary schools. J Human Nutr Diet. 2019;32(1):31–40. 10.1111/jhn.12598 [DOI] [PubMed] [Google Scholar]
  • 72. Taylor JP, McKenna ML, Butler GP: Monitoring and evaluating school nutrition and physical activity policies. Can J Public Health. 2010;101 Suppl 2(Suppl 2):S24–7. 10.1007/BF03405622 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73. Nilsen P: Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53. 10.1186/s13012-015-0242-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74. Clarke AM, O’ Sullivan M, Barry MM: Context matters in programme implementation. Health Ed. 2010;110(4):273–93. 10.1108/09654281011052637 [DOI] [Google Scholar]
  • 75. Durlak JA, DuPre EP: Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50. 10.1007/s10464-008-9165-0 [DOI] [PubMed] [Google Scholar]
  • 76. Lynch EA, Mudge A, Knowles S, et al. : “There is nothing so practical as a good theory”: a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. 2018;18(1):857. 10.1186/s12913-018-3671-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77. Ronto R, Rathi N, Worsley A, et al. : Enablers and barriers to implementation of and compliance with school-based healthy food and beverage policies: a systematic literature review and meta-synthesis. Public Health Nutr. 2020;23(15):2840–2855. [published online ahead of print, 2020 Apr 22]. 10.1017/S1368980019004865 [DOI] [PMC free article] [PubMed] [Google Scholar]
HRB Open Res. 2021 Jan 20. doi: 10.21956/hrbopenres.14342.r28560

Reviewer response for version 2

Katie A Weatherson 1

Thank you for your thoughtful consideration of the first review. You have satisfactorily addressed my previous comments. I have some minor additional comments for your consideration:

  1. Abstract - last sentence - adaptation or adoption?

  2. Analysis - Step 1: Data Extraction section - You still do not describe how you will identify and extract barriers and facilitators from included articles. E.g. How will you identify a 'data point'? If qualitative, how much text will you extract? You seem to include extraction details in Step 2. I would suggest parsing out if in fact two separate steps.

  3. General grammatical errors throughout (spelling, tense, incorrect use of semicolons/colons). I would suggest reading thoroughly to identify mistakes.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

My research primarily focuses on the implementation and effectiveness evaluation of initiatives aiming to improve the health of populations. Specifically related is a review of factors influencing the implementation of physical activity policies in schools.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

HRB Open Res. 2020 Jul 20. doi: 10.21956/hrbopenres.14137.r27547

Reviewer response for version 1

Sze Lin Yoong 1

Thank you for the opportunity to review this protocol. It is clear and well-conducted.

I have provided some minor comments below for the authors to consider:

  1. In the introduction, it may be worth including information on the lack of implementation of food-based guidelines routinely to highlight the challenges with implementing these guidelines and thus enforce the rationale for this review.

  2. Some of the rationales for using the TDF may not be relevant to the study. For example, you say that using the TDF will ensure a range of factors examined, but this is only relevant if it is examined in the primary study. Perhaps it is sufficient here to say that using the TDF to synthesis findings provides you with a way of identifying theoretical constructs to target in the development of interventions.

  3. The review has two aims – the second aim is not clearly addressed in the methods and synthesis. Are you planning to do this in a subgroup analysis?

  4. Repopulation – will you include vocational/technical schools (post-secondary schools) or is this excluded?

  5. You say that studies including children’s perceived barriers will be excluded – is this because these are demand focused interventions. Or that children are not involved in implementing food standards? Please provide a rationale.

  6. Can you provide a definition or specific example of what constitutes a food standard?

  7. The search string looks reasonable although I’m not sure you need a third category (for school only) if your search #2 includes school in most of the search terms. Will you be exploring your mesh headings?

  8. Can you provide more information about coding and extraction according to the TDF and how that will be done?

  9. Can you add how the second aim will be analysed/explore? Are you exploring barriers by different subgroups? Are you describing your finding narratively or in tabular form?

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Implementation science, child nutrition, systematic review

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

HRB Open Res. 2020 Dec 7.
Breda O'Mahony 1

Dear Dr. Yoong,

Thank you for agreeing to be part of the peer review process for this manuscript. We are grateful for all your comments and suggestions on our manuscript. We have carefully considered the comments and have responded to each comment below.

We hope that our response has provided some clarity which will ultimately improve the quality of the manuscript.

Yours Sincerely,

Breda O’ Mahony

Comment 1: Thank you for the opportunity to review this protocol. It is clear and well-conducted.

I have provided some minor comments below for the authors to consider.

Response 1: We are grateful to the reviewer for their positive comments in relation to the approach for this systematic review. We also appreciate the time that the reviewer gave regarding suggestions and amendments. We hope that we have answered these comments with sufficient detail.

Comment 2: In the introduction, it may be worth including information on the lack of implementation of food-based guidelines routinely to highlight the challenges with implementing these guidelines and thus enforce the rationale for this review.

Response 2: This is a valid suggestion. We have revised paragraph five in the ‘background’ section to include this suggestion.

It has been noted that more support must be provided to schools to allow implementation of nutritional guidelines (Abery and Drummond, 2014). Data from Gregoric and colleagues (2015) found that the implementation of school nutrition guidelines were achieved differently at distinct levels.

Reference:

Abery E, and Drummond, C. (2014) "Implementation of Mandatory Nutritional Guidelines in South Australian Primary School Canteens: A Qualitative Study."  Improving Schools 17.1: 41-53.

Comment 3: Some of the rationales for using the TDF may not be relevant to the study. For example, you say that using the TDF will ensure a range of factors examined, but this is only relevant if it is examined in the primary study. Perhaps it is sufficient here to say that using the TDF to synthesis findings provides you with a way of identifying theoretical constructs to target in the development of interventions.

Response 4: Thank you for highlighting this. We have implemented your feedback in the updated manuscript to include:  ‘Using the TDF to synthesize findings provides a mechanism to identify theoretical constructs to target in the development of food standards/guideline implementation’.

Comment 4: The review has two aims – the second aim is not clearly addressed in the methods and synthesis. Are you planning to do this in a subgroup analysis?

Response 4: We did not explicitly commit to this comparison as it is dependent on the number of studies retrieved. To allow comparison, subgroup analysis will be performed. This will be completed post hoc, so as to allow the authors to assess if this is an achievable aim. Work by Oxman and Guyatt   (1992)   and Yusuf et al.   (1991) will be used to guide subgroup analyses. Our approach to this is included under response 9.

References:

Oxman AD, Guyatt GH. A consumer's guide to subgroup analyses.  Ann Intern Med. 1992;116(1):78-84. doi:10.7326/0003-4819-116-1-78

Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and Interpretation of Treatment Effects in Subgroups of Patients in Randomized Clinical Trials.  JAMA. 1991;266(1):93–98. doi:10.1001/jama.1991.03470010097038

Comment 5: Re population – will you include vocational/technical schools (post-secondary schools) or is this excluded?

Response 5: All variations of primary and post primary school will be included. This includes, but is not limited to: high school, junior school, middle school, public school. Pre-primary and post-secondary schools will be excluded.

This will be added to the updated manuscript in the ‘population’ section.

Comment 6: You say that studies including children’s perceived barriers will be excluded – is this because these are demand focused interventions. Or that children are not involved in implementing food standards? Please provide a rationale.

Response 6: As you noted, children are not included because they are demand-side stakeholder.  This systematic review is focused specifically on supply side stakeholders rather than demand-side stakeholders. However, we acknowledge that student’s perceptions and experiences are key, as they are at the core of the standards. Upon completion of this systematic review, the authors intend to undertake research on student’s experiences of the food standards.

Comment 7: Can you provide a definition or specific example of what constitutes a food standard?

Response 7: Nutrient standards are based on limits and promotions of various nutrients whereas food based standards set requirements on what food can and cannot be served and how frequently (Haroun et al., 2011).

For clarity, the terms food and nutrient standard will be described in the revised manuscript (Paragraph 3).

Reference:

Haroun, Dalia & Wood, Lesley & Harper, Clare & Nelson, Michael. (2011). Nutrient-based standards for school lunches complement food-based standards and improve pupils' nutrient intake profile. The British journal of nutrition. 106. 472-4. 10.1017/S0007114511002297.

Comment 8: The search string looks reasonable although I’m not sure you need a third category (for school only) if your search #2 includes school in most of the search terms. Will you be exploring your mesh headings?

Response 8: Author one has been trialling various databases searches and looking at other reviews in this area to inform the search string. The addition of education* along with school* has been trialled. To date, this pilot search have produced positive results. MeSH headings are included on the applicable databases.

Comment 9: Can you provide more information about coding and extraction according to the TDF and how that will be done?

Response 9: Based on other reviewer comments, we have re-worked the section on data extraction and data synthesis so we avoid repetition but still ensure our comprehensive approach is clear. Further details in relation to coding and extraction are included in the following paragraphs.

Data extraction and analysis

For data extraction and synthesis, we have adopted a similar approach undertaken in a study protocol by Graham-Rowe et al. (2016). Data will be extracted from all full text studies that meet the inclusion criteria, using a data extraction form. The data extraction form will be created using Microsoft Word. Data to be extracted will include, but is not limited to, the following: Key study information which will include study title, name of the first author, year of publication, country of study, language, study type (qualitative, quantitative and mixed methods studies), study design (e.g. cross sectional and observational), intervention type (e.g. food standards, nutrient standards), type of implementation  (voluntary or mandatory participation), educational setting (primary, post primary, academies etc.) and participant characteristics (canteen staff, head teacher, contracted catering suppliers, food service directors and managers, programme coordinators etc.), sample size, data collection and analysis methods. Data on intervention effects/outcomes, such as change in children’s dietary habits will not form part of this review. A coding manual with the definitions outlined by Cane et al, for the 14 theoretical domains from the TDF, will also be prepared. 

Analysis

Step 1: Data extraction.

Two review authors, not blind to author or journal information will independently extract from all the full text studies that fulfil the inclusion criteria, using a data extraction form. Each data point will be categorised as either (a) raw data (e.g. quotes from qualitative studies); (b) analysed data from results sections (e.g. statistical analysis from surveys and questionnaires (c) interpretative descriptions and summaries of results from published reports.

Deductive analysis

Step 2: Coding of extracted data

  1. Pilot of the coding of data

To ensure validity and reliability, a pilot be will undertaken to allow the review authors to practice coding extracted data into the TDF domains. This will enable comparison of data coding and implement modification, if required. This will be jointly undertaken by the two review authors using the domain definition manual.  Any discrepancies will be resolved by the review team, and if necessary by a third reviewer.

  1. Coding extracted data to the TDF

The lead author will continue to code the extracted data, using the domain definition manual, to the TDF domain that they judge to best represent the factor. For qualitative studies, both the theme and the original reporting format of the barrier or facilitator will be extracted. This will include participant quotes, excerpts, quotations or entire passages from studies using documentary analysis, and narrative descriptive summaries of results. If a reported barrier or facilitator is judged to represent more than one domain, it will be coded multiply into its associated domains. Where a factor is not recorded as a barrier or facilitator, we will revert back to the definitions, as previously stated. For quantitative studies, the frequency of all  the barriers and facilitators will be extracted and placed in the correct TDF construct. This is important so as to be able to highlight the pertinent factors affecting implementation and provide relevant recommendations. Where mixed method studies are included, we envisage that two data extractions will be required. This ensures all factors will be documented. To avoid overlap, data will be extracted for the qualitative data sheet and for the quantitative data sheet. To access coding reliability, a second review author will assess a random sample of 20%.Kappa statistics will be used again to assess inter-rater reliability, with values of 0.75 and higher representing high agreement.

Inductive analysis

Step 3: Thematic synthesis

Inductive thematic synthesis, based on methods by Thomas and Harden (2008) will also be used to code any data that does not fit into the TDF. This will require a three step process. Step one involves line by line coding of the data that was not coded to the TDF. The second step involves organisation or grouping of these code into associated areas to construct descriptive themes. In step three, the descriptive themes will be compared to refine the relationship between them so as to generate analytical themes.

Sensitivity analysis

Following the synthesis stage, the first author will perform a sensitivity analysis. This will determine if the review findings are sensitive to the following variables: study quality, study methodology (qualitative, quantitative or mixed methods) and location. Both bias and sensitivity reviews will help ensure quality, rigour and transparency (Langer et al, 2015).

Importance criterion

To determine which domains are likely to be the most important factors influencing implementation, each domain in step two will be reviewed. In this review, the frequency of each domain will be examined. Similar to a study protocol by Graham-Rowe et al. (2016)., for qualitative studies we will then consider ‘expressed importance’ within each domain by looking for a statement(s) from the authors interpretation of the study findings articulating what beliefs were reported to be important by the study participants. We acknowledge that this is not a precise process but it has a good fit with the qualitative approach as the meaning, interpretation and prioritisation of the data will be by the authors who have closer familiarity with the primary data, than will be possible by the review team. This process will require discussion and critical judgement by the research team. This will allow the review team, to interpret the domains that have the highest frequency and ‘expressed importance’ as the most important factors in the implementation of school meals standards/guidelines.

If there is sufficient data, we will explore whether the domains identified as important vary according to the educational establishment i.e. primary and post primary schools. This will be performed by subgroup analysis. Findings will be described in tabular format.

References:

Graham-Rowe, E., et al. (2016). "Barriers and enablers to diabetic retinopathy screening attendance: Protocol for a systematic review." Systematic Reviews 5(1): 134.

Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews.  BMC Med Res Methodol. 2008; 8: 45. 

Langer L, Stewart R, Winters N. Sensitivity analysis in systematic reviews of qualitative evidence: an example from a mixed-methods systematic review . In: Filtering the information overload for better decisions. Abstracts of the 23rd Cochrane Colloquium; 2015 3-7 Oct; Vienna, Austria. John Wiley & Sons; 2015.

Comment 10: Can you add how the second aim will be analysed/explore? Are you exploring barriers by different subgroups? Are you describing your finding narratively or in tabular form?

Response 10: This is now included under response 9 and in the section ‘data extraction and synthesis’.

If there are sufficient data, we will explore whether the domains identified as important vary according to the educational establishment i.e. primary and post primary schools. Additionally, if there are sufficient data to compare views across the different supply side stakeholders, this will also be explored. Findings will be described in tabular format.

HRB Open Res. 2020 Jul 13. doi: 10.21956/hrbopenres.14137.r27563

Reviewer response for version 1

Katie A Weatherson 1

The review will address an important question- how to improve the implementation of food and nutrition standards/policies in schools. In general, the study protocol is clearly written and the use of the TDF to categorize barriers and facilitators is a useful addition. I have the following points to clarify the study and overall methodology:

Abstract:

 

  • Keywords: suggest adding ‘nutrition’

 

Background:

  • Clarify “their”. E.g., Children, youth.

  • Be consistent with use of term “school-based” and “supply-side”. In some places you do not use hyphens.

  • Paragraph 3 – This section would benefit from a few concrete details of existing nutrition or food standards for school food (meals/snacks). I suggest adding a couple of examples.

  • Add citation: “…difficulties associated with preparing and serving fresh food at school;”

  • Paragraph 6 – suggest adding definition of ‘implementation’.

  • Add citation : “…later validated in 2012 (version 2 (v2)).”

 

Methods: 

  • Inclusion criteria – does this include standards for snacks sold in vending machines at school or breakfast programs?

  • “The findings will include the following: (i) verbatim quotations from research participants; (ii) excerpts, quotations or entire passages from studies” – change to “may include”

  • Outcome sub-section, last sentence: this is a secondary outcome of your review, not of the studies you are including in your review. Revise.

  • “The lead author will screen study titles and abstracts to decide whether the full text manuscript should be retrieved. For screening reliability, a second review author will assess a random sample of 20%. Kappa statistics will be used to assess inter-rater reliability, with values of 0.75 and higher representing high agreement.” What will happen in cases of disagreement? Full article will be screened?

  • Data extraction sub-section: Revise. You will be extracting barriers and facilitators, NOT a coding manual.  “Data to be extracted will include, … (b) a coding manual with a definition for each of the 14 TDF constructs…”.

  • Also, revise “study design (education setting/school type).” Education setting/school type does not reflect study design. Based on your secondary outcome, you should be extracting Education setting (e.g., primary or post primary) data.

  • Please provide a rationale for why barriers and facilitators will be extracted from both the results and the discussion sections of included articles. It seems a more valid approach would be to only extract data from the results section, as the discussion section is where authors compare their findings in light of other literature. Extracting information from the discussion would likely lead to double extraction of the same barrier/facilitator (and thus greater emphasis placed on these factors), and/or errors in extraction (extracting results from other comparative studies).

  • I also suggest adding more specific details about extraction for qualitative vs quantitative studies. For example, in qualitative studies, results are often presented as themes, with individual quotes used to highlight the theme. Will both the theme and the individual quotes be extracted? In quantitative studies, will all findings be extracted, irrespective of how many participants agreed/reported that the barrier/facilitator existed (i.e., frequency)? I see now that you have reported this under data synthesis – however, I suggest adding to data extraction sub-section. It is important to think through how extracted data will be quantified in your review.

  • Quality assessment sub-section: the last sentence does not make sense.

  • Data synthesis sub-section: I recommend adding specific mention of the coding manual in this section. Will only one author code the extracted barriers/facilitators to the TDF domains?

 

Discussion:

 

  • Paragraph 3 – Add sentence about specific evidence gap before referring to it in sentence two.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

My research primarily focuses on the implementation and effectiveness evaluation of initiatives aiming to improve the health of populations. Specifically related is a review of factors influencing the implementation of physical activity policies in schools.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

HRB Open Res. 2020 Dec 7.
Breda O'Mahony 1

Dear Dr. Weatherson,

Thank you for agreeing to be part of the peer review process for this manuscript. We are grateful for all your comments and suggestions on our manuscript. We have carefully considered the comments and have responded to each comment below.

We hope that our response has provided come clarity which will ultimately improve the quality of the manuscript.

Yours Sincerely,

Breda O’ Mahony

Comment 1: The review will address an important question- how to improve the implementation of food and nutrition standards/policies in schools. In general, the study protocol is clearly written and the use of the TDF to categorize barriers and facilitators is a useful addition. I have the following points to clarify the study and overall methodology:

Response 1: We are grateful to the reviewer for their positive comments in relation to the concept and rationale for this systematic review. We also appreciate the time that the reviewer gave regarding suggestions and amendments.

Abstract

Comment 2: Abstract: Keywords: suggest adding ‘nutrition’

Response 2: HRB Open guidelines state that up to eight keywords can be used. We have removed the key word ‘schools’ and replaced it with ‘nutrition’. We made this swap as school is represented in keywords ‘school meal standards’ and ‘school food guidelines’.

Background

Comment 3: Clarify “their”. E.g., Children, youth.

Response 3: We have removed the word ‘their’ and replaced this with ‘child and adolescent’. Paragraph 1 now reads: Schools are a key setting for the promotion of health and well-being 1 – 3 . They are one of the most effective ways of reaching a large segment of the population 4 , 5 , with no other institution having as much continuous contact and influence during the first stages of a child life 6 .

Comment 4: Be consistent with use of term “school-based” and “supply-side”. In some places you do not use hyphens.

Response 4: Thank you for pointing out this. We will ensure that the use of hyphens is consistent in the updated version (version 2).

Comment 5: Paragraph 3 – This section would benefit from a few concrete details of existing nutrition or food standards for school food (meals/snacks). I suggest adding a couple of examples.

Response 5: Thank you for this suggestion. This paragraph has now been extended to:

To date, a number of countries and regions around the world have introduced nutrition or food standards for school food on a mandatory basis. These include Finland in 1943, Sweden in 1997 , Norway in 2001, a reintroduction of compulsory guidelines after 21 years in England in 2001, Slovenia in 2010, and an updated National School Lunch Programme in United States of America (USA) in 2012, which will be phased into all schools by 2023. Nutrient standards are based on limits and promotions of various nutrients whereas food based standards set requirements on what food can and cannot be served and how frequently (Haroun et al., 2011).  Examples of standards include the Irish Nutrition Standards for School meals (Department of Health, 2017). The food based standards set out requirements for each meal type e.g. for breakfast a minimum of two items should be provided, one serving of wholemeal or wholegrain cereals and breads, and one serving of either milk, yoghurt or cheese or fruit.

Comment 6: Add citation: “…difficulties associated with preparing and serving fresh food at school;”

Response 6:  Citation has been added and will be included to the updated version.  It will read “…difficulties associated with preparing and serving fresh food at school 20;”

20. Tabak RG, Moreland-Russell S: Food Service Perspectives on National School Lunch Program Implementation.  Health Behav and Policy Rev. 2015;  2(5): 362–371

Comment 7: Paragraph 6 – suggest adding definition of ‘implementation’.

Response7: Thank you for this suggestion, we added an explanation of ‘implementation’ to paragraph 5. This will be added to the updated version and paragraph 5 will now read:

It has been noted that more support must be provided to schools to allow implementation of nutritional guidelines (Abery and Drummond, 2014). Data from Gregoric and colleagues (2015) 13 found that the implementation of school nutrition guidelines differed by schools within the same region. Implementation as a process and a science is complex to study because of the numerous factors affecting implementation, its process, results and potential solutions (Peters et al, 2014).

Reference:

Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. (2014) "Republished research: Implementation research: what it is and how to do it." British Journal of Sports Medicine 48(8): 731.

Comment 8: Add citation : “…later validated in 2012 (version 2 (v2)).”

Response 8: Thank you for highlighting this omission; a citation has now been added “…later validated in 2012 (version 2 (v2)) 36.”

Reference:

36. Atkins, L., Francis, J., Islam, R.  et al. A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems.  Implementation Sci  12, 77 (2017)

Methods

Comment 9: Inclusion criteria – does this include standards for snacks sold in vending machines at school or breakfast programs?

Response 9: Yes, this includes standards for snacks sold in vending machines at school and or school breakfast programs. This will be made more explicit in the updated manuscript, under paragraph ‘Intervention’.

Comment 10: “The findings will include the following: (i) verbatim quotations from research participants; (ii) excerpts, quotations or entire passages from studies” – change to “may include”

Response 10: This change will be made to the updated version and will read “The findings may include the following: (i) verbatim quotations from research participants; (ii) excerpts, quotations or entire passages from studies”.

Comment 11: Outcome sub-section, last sentence: this is a secondary outcome of your review, not of the studies you are including in your review. Revise

Response 11: This will now be revised to:

A secondary outcome of the review may include comparing the barriers and facilitators between a primary and post primary school setting.

Comment 12: “The lead author will screen study titles and abstracts to decide whether the full text manuscript should be retrieved. For screening reliability, a second review author will assess a random sample of 20%. Kappa statistics will be used to assess inter-rater reliability, with values of 0.75 and higher representing high agreement.” What will happen in cases of disagreement? Full article will be screened?

Response 12: In the case of disagreement, full article will be screened. This will be added to the revised manuscript.

Comment 13: Data extraction sub-section: Revise. You will be extracting barriers and facilitators, NOT a coding manual.  “Data to be extracted will include, … (b) a coding manual with a definition for each of the 14 TDF constructs…”.

Response 13: Thank you for noticing this error. This has now been removed and added instead to the data extraction and synthesis paragraph.

Comment 14: Also, revise “study design (education setting/school type).” Education setting/school type does not reflect study design. Based on your secondary outcome, you should be extracting Education setting (e.g., primary or post primary) data.

Response 14: Apologies for the error. The text will be amended to education setting (e.g. primary or post primary) in the updated manuscript.

Comment 15: Please provide a rationale for why barriers and facilitators will be extracted from both the results and the discussion sections of included articles. It seems a more valid approach would be to only extract data from the results section, as the discussion section is where authors compare their findings in light of other literature. Extracting information from the discussion would likely lead to double extraction of the same barrier/facilitator (and thus greater emphasis placed on these factors), and/or errors in extraction (extracting results from other comparative studies).

Response 15: The decision to include the discussion sections (with the results sections) is for qualitative studies only. Some journals have limited word counts so the inclusion of author interpretations can ensure findings are not omitted. Thus, the inclusion of author’s interpretations in qualitative studies allows for a more in-depth syntheses.

This will be more explicit in the updated manuscript.

Comment 16: I also suggest adding more specific details about extraction for qualitative vs quantitative studies. For example, in qualitative studies, results are often presented as themes, with individual quotes used to highlight the theme. Will both the theme and the individual quotes be extracted? In quantitative studies, will all findings be extracted, irrespective of how many participants agreed/reported that the barrier/facilitator existed (i.e., frequency)? I see now that you have reported this under data synthesis – however, I suggest adding to data extraction sub-section. It is important to think through how extracted data will be quantified in your review.

Response 16: We have taken your comment on board and we have revised the data extraction paragraph to take on board all of the reviewers’ comments and suggestions. This will now read:

Data extraction and analysis materials/tools

For data extraction and synthesis, we have adopted a similar approach undertaken in a study protocol by Graham-Rowe et al. (2016). Data will be extracted from all full text studies that meet the inclusion criteria, using a data extraction form. The data extraction form will be created using Microsoft Word. Data to be extracted will include, but is not limited to, the following: Key study information which will include study title, name of the first author, year of publication, country of study, language, study type (qualitative, quantitative and mixed methods studies), study design (e.g. cross sectional and observational), intervention type (e.g. food standards, nutrient standards), type of implementation  (voluntary or mandatory participation), educational setting (primary, post primary, academies etc.) and participant characteristics (canteen staff, head teacher, contracted catering suppliers, food service directors and managers, programme coordinators etc.), sample size, data collection and analysis methods. Data on intervention effects/outcomes, such as change in children’s dietary habits will not form part of this review. A coding manual with the definitions outlined by Cane et al, for the 14 theoretical domains from the TDF, will also be prepared. 

Analysis

Step 1: Data extraction.

Two review authors, not blind to author or journal information will independently extract from all the full text studies that fulfil the inclusion criteria, using a data extraction form. Each data point will be categorised as either (a) raw data (e.g. quotes from qualitative studies); (b) analysed data from results sections (e.g. statistical analysis from surveys and questionnaires (c) interpretative descriptions and summaries of results from published reports.

Deductive analysis

Step 2: Coding of extracted data

  1. Pilot of the coding of data

To ensure validity and reliability, a pilot be will undertaken to allow the review authors to practice coding extracted data into the TDF domains. This will enable comparison of data coding and implement modification, if required. This will be jointly undertaken by the two review authors using the domain definition manual.  Any discrepancies will be resolved by the review team, and if necessary by a third reviewer.

  1. Coding extracted data to the TDF

The lead author will continue to code the extracted data, using the domain definition manual, to the TDF domain that they judge to best represent the factor. For qualitative studies, both the theme and the original reporting format of the barrier or facilitator will be extracted. This will include participant quotes, excerpts, quotations or entire passages from studies using documentary analysis, and narrative descriptive summaries of results. If a reported barrier or facilitator is judged to represent more than one domain, it will be coded multiply into its associated domains. Where a factor is not recorded as a barrier or facilitator, we will revert back to the definitions, as previously stated. For quantitative studies, the frequency of all  the barriers and facilitators will be extracted and placed in the correct TDF construct. This is important so as to be able to highlight the pertinent factors affecting implementation and provide relevant recommendations. Where mixed method studies are included, we envisage that two data extractions will be required. This ensures all factors will be documented. To avoid overlap, data will be extracted for the qualitative data sheet and for the quantitative data sheet. To access coding reliability, a second review author will assess a random sample of 20%.Kappa statistics will be used again to assess inter-rater reliability, with values of 0.75 and higher representing high agreement.

Inductive analysis

Step 3: Thematic synthesis

Inductive thematic synthesis, based on methods by Thomas and Harden (2008) will also be used to code any data that does not fit into the TDF. This will require a three step process. Step one involves line by line coding of the data that was not coded to the TDF. The second step involves organisation or grouping of these code into associated areas to construct descriptive themes. In step three, the descriptive themes will be compared to refine the relationship between them so as to generate analytical themes.

Sensitivity analysis

Following the synthesis stage, the first author will perform a sensitivity analysis. This will determine if the review findings are sensitive to the following variables: study quality, study methodology (qualitative, quantitative or mixed methods), type of implementation (voluntary or mandatory participation) and location. Both bias and sensitivity reviews will help ensure quality, rigour and transparency (Langer et al, 2015)

Importance criterion

To determine which domains are likely to be the most important factors influencing implementation, each domain in step two will be reviewed. In this review, the frequency of each domain will be examined. Similar to a study protocol by Graham-Rowe et al. (2016)., for qualitative studies we will then consider ‘expressed importance’ within each domain by looking for a statement(s) from the authors interpretation of the study findings articulating what beliefs were reported to be important by the study participants. We acknowledge that this is not a precise process but it has a good fit with the qualitative approach as the meaning, interpretation and prioritisation of the data will be by the authors who have closer familiarity with the primary data, than will be possible by the review team. This process will require discussion and critical judgement by the research team. This will allow the review team, to interpret the domains that have the highest frequency and ‘expressed importance’ as the most important factors in the implementation of school meals standards/guidelines.

If there is sufficient data, we will explore whether the domains identified as important vary according to the educational establishment i.e. primary and post primary schools. This will be performed by subgroup analysis. Findings will be described in tabular format.

References:

Graham-Rowe, E., et al. (2016). "Barriers and enablers to diabetic retinopathy screening attendance: Protocol for a systematic review." Systematic Reviews 5(1): 134.

Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews.  BMC Med Res Methodol. 2008; 8: 45. 

Langer L, Stewart R, Winters N. Sensitivity analysis in systematic reviews of qualitative evidence: an example from a mixed-methods systematic review . In: Filtering the information overload for better decisions. Abstracts of the 23rd Cochrane Colloquium; 2015 3-7 Oct; Vienna, Austria. John Wiley & Sons; 2015.

Comment 17: Quality assessment sub-section: the last sentence does not make sense.

Response 17: This sentence will now read “When disagreements between reviewers cannot be easily resolved, a third independent reviewer will become involved in order to reach consensus.

Comment 18: Data synthesis sub-section: I recommend adding specific mention of the coding manual in this section. Will only one author code the extracted barriers/facilitators to the TDF domains?

Response 18: We have expanded the information under extraction and analyses. This includes more specific information on the TDF. This paragraph will now read:

Deductive analysis

Step 2: Coding of extracted data

  1. Pilot of the coding of data

To ensure validity and reliability, a pilot be will undertaken to allow the review authors to practice coding extracted data into the TDF domains. This will enable comparison of data coding and implement modification, if required. This will be jointly undertaken by the two review authors using the domain definition manual.  Any discrepancies will be resolved by the review team, and if necessary by a third reviewer.

  1. Coding extracted data to the TDF

The lead author will continue to code the extracted data, using the domain definition manual, to the TDF domain that they judge to best represent the factor. For qualitative studies, both the theme and the original reporting format of the barrier or facilitator will be extracted. This will include participant quotes, excerpts, quotations or entire passages from studies using documentary analysis, and narrative descriptive summaries of results. If a reported barrier or facilitator is judged to represent more than one domain, it will be coded multiply into its associated domains. Where a factor is not recorded as a barrier or facilitator, we will revert back to the definitions, as previously stated. For quantitative studies, the frequency of all  the barriers and facilitators will be extracted and placed in the correct TDF construct. This is important so as to be able to highlight the pertinent factors affecting implementation and provide relevant recommendations. Where mixed method studies are included, we envisage that two data extractions will be required. This ensures all factors will be documented. To avoid overlap, data will be extracted for the qualitative data sheet and for the quantitative data sheet. To access coding reliability, a second review author will assess a random sample of 20%.Kappa statistics will be used again to assess inter-rater reliability, with values of 0.75 and higher representing high agreement.

Discussion

Comment 19: Paragraph 3 – Add sentence about specific evidence gap before referring to it in sentence two.

Response 19: Paragraph 3 under the Discussion section will now read-

A number of studies have identified various factors that affect implementation, however there has been limited synthesis of this information from a stakeholder’s perspective. Ronto and colleagues (2020) recent systematic review analysed the implementation and/or compliance with school-based healthy food and beverage policies. The eligibility criteria included policy relating to nutrition guidelines, regulations and/or restrictions on food and beverage availability, advertisement, placement or price. There was no exclusion criteria in relation to study participants, but grey literature was not accessed. Our mixed methods review is specific to the implementation of food/nutrient based standards from a stakeholder’s perspective. To ensure a comprehensive review is undertaken, a variety of electronic databases and grey literature sources will be assessed. Given the potential impact that school meal guidelines and standards can have on the health and wellbeing outcomes of children and adolescents, understanding the factors that affect their implementation is key. We are confident that the depth of this review will provide a holistic understanding of the factors as all types of studies; qualitative and quantitative or both, including grey literature, will be accessed. Furthermore, there will no language or publication date restrictions. The review will follow academic rigour and will include a number of strategies for validity, reliability and to reduce the effects of bias. This will be achieved by having clear and detailed inclusion and exclusion criteria, independent reviewers, the use of PRISMA guidelines, a MMAT, and by using computer packages for data and quality management. Finally, where deviations from this protocol occur, this will be justified and discussed in the systematic review upon publication and will be documented on PROSPERO.

HRB Open Res. 2020 May 21. doi: 10.21956/hrbopenres.14137.r27383

Reviewer response for version 1

Suzanne Spence 1

This is an interesting and well thought through concept for a systematic review. I am not an expert in systematic review methodologies and therefore cannot comment in great detail on that aspect. Overall, the rationale is clear, I have a few wider comments for the authors to consider for this review.

Objectives:

  • The authors have a primary objective to identify and synthesise the existing evidence on the barriers and facilitators to implementing food or nutrition standards for school food from supply-side stakeholders. Would you please clarify the term supply-side stakeholders: does this include school caterers, headteachers, school business managers, and canteen staff too?

  • The authors also mention a secondary objective may be included to compare the barriers and facilitators between a primary and post-primary school setting. I think this objective should be included from the outset - primary schools are very different settings to secondary schools/academies and will be an important aspect of the review to reflect this in the analysis and discussions.

 

Population:

The authors are excluding studies that involve school children’s perceived barriers and facilitators – is it possible to give a rationale for excluding children’s perceptions?

Intervention/Methods:

If possible, if the authors could provide some clarification on the following points about the intervention that would be helpful. The authors note they will include studies:

  • Where interventions delivered in educational establishments and school food standards have been implemented on a voluntary or mandatory basis.

  • No restriction on the type of standard i.e. nutrient and food-based standards will be included

  • And the international span of countries

How do the authors intend to explore/take into account the effect of the difference between voluntary and mandatory on barriers/success of implementation – these are quite different approaches to implementation.

Similarly, food and nutrient-based standards vary globally re requirements/complexity – how will country effect be captured in review? Both complexity of standards and variation by country may well influence the barriers/success of the implementation of standards.

While there is a clear overview/outcome of data extraction for the qualitative component I am unclear about the outcome for the quantitative aspect, is it an effect on nutrition, children’s diets, etc. A bit more clarification for the quantitative outcome is required.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Public Health Nutrition, children's dietary intake, policy evaluation of school food and nutrient standards

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

HRB Open Res. 2020 Jun 17.
Breda O'Mahony 1

Dear Dr. Spence,

Thank you for agreeing to be part of the peer review process for this manuscript. We are grateful for all your comments and suggestions on our manuscript. We have carefully considered the comments and have responded to each comment below.

We hope that our response has provided come clarity which will ultimately improve the quality of the manuscript.

Yours Sincerely,

Breda O’ Mahony

Comment 1

This is an interesting and well thought through concept for a systematic review. I am not an expert in systematic review methodologies and therefore cannot comment in great detail on that aspect. Overall, the rationale is clear, I have a few wider comments for the authors to consider for this review.

Response 1

Thank you for your positive comments in relation to the concept and rationale for this systematic review.

Comment 2

Objective 1: The authors have a primary objective to identify and synthesise the existing evidence on the barriers and facilitators to implementing food or nutrition standards for school food from supply-side stakeholders. Would you please clarify the term supply-side stakeholders: does this include school caterers, headteachers, school business managers, and canteen staff too?

Response 2

The reviewer’s interpretation of supply side stakeholder is correct. The term supply side stakeholders refers, but is not limited to catering management and staff, school principals/managers, contracted catering suppliers, food service directors and managers, programme coordinators (this is outlined under paragraph ‘Population’ on page 4). Studies involving school children’s perceived barriers and facilitators will not be included. Furthermore, standards in pre-schools and third level settings will also be excluded.

Comment 3

Objective 2: The authors also mention a secondary objective may be included to compare the barriers and facilitators between a primary and post-primary school setting. I think this objective should be included from the outset - primary schools are very different settings to secondary schools/academies and will be an important aspect of the review to reflect this in the analysis and discussions.

Response 3

Thank you for this suggestion. We did not explicitly commit to this comparison as it is dependent on the number of studies retrieved. The manuscript will be amended and will read: “A secondary objective is to compare the barriers and facilitators between a primary and post primary school setting”.

Comment 4

Population: The authors are excluding studies that involve school children’s perceived barriers and facilitators – is it possible to give a rationale for excluding children’s perceptions?

Response 4

This is a valuable point, however we believe that this would greatly expand the focus of this systematic review. This systematic review is focused on supply side stakeholders rather than demand-side stakeholders (i.e. students). However, we acknowledge that student’s perceptions and experiences are key, as they are at the core of the standards. Upon completion of this systematic review, the authors intend to undertake research on student’s experiences of the food standards. 

Comment 5

Intervention

If possible, if the authors could provide some clarification on the following points about the intervention that would be helpful. The authors note they will include studies:

Where interventions delivered in educational establishments and school food standards have been implemented on a voluntary or mandatory basis.

Response 5

The authors aim to capture all internationally suitable research that relates to the implementation of school food standards. After carrying out a preliminary scoping search, it illustrated that some schools implemented food standards on a voluntary or mandatory basis, e.g. in response to national or regional policy. The scoping search also identified studies where school food standards were voluntarily implemented for research purposes. Such studies will also be included.

Comment 6

If possible, if the authors could provide some clarification on the following points about the intervention that would be helpful. The authors note they will include studies:

No restriction on the type of standard i.e. nutrient and food-based standards will be included

Response 6

After conducting a preliminary search to inform the protocol and systematic review, we found that some jurisdictions used the term nutrient standards (or similar deviations) whilst other jurisdictions used the term food based standards (or similar deviations). Both terms are relevant as they focus on food provision. Nutrient standards are based on limits and promotions of various nutrients whereas food based standards set requirements on what food can and cannot be served and how frequently (Haroun et al., 2011). To ensure all suitable studies are captured, this has been reflected in the eligibility study criteria for inclusion/exclusion. Our eligibility criteria clearly rules out studies based on dietary guidelines, healthy eating policies etc.

Comment 7

If possible, if the authors could provide some clarification on the following points about the intervention that would be helpful. The authors note they will include studies:

And the international span of countries

Response 7

The authors decided rather than selecting certain jurisdictions, to expand and capture all suitable international research on nutrient or food based standards. For this reason, we decided not to apply an English language restriction.

Comment 8

How do the authors intend to explore/take into account the effect of the difference between voluntary and mandatory on barriers/success of implementation – these are quite different approaches to implementation.

Response 8

Due to similarities in responses between comment 8 & 9, the response to comment 8 will be outlined under response 9.

Comment 9

Similarly, food and nutrient-based standards vary globally re requirements/complexity – how will country effect be captured in review? Both complexity of standards and variation by country may well influence the barriers/success of the implementation of standards.

Response 9 

The author’s acknowledge the implementation is context dependent and have noted to ensure that this forms part of the key information. As part of data extraction and testing of the synthesis, the authors will include the intervention type (i.e. nutrient, food standard) and type of implementation (intervention, voluntary or mandatory participation) and study setting (primary or post primary) as part of data extraction and testing of the synthesis.

The authors also recognise that different types of supply-side stakeholders (i.e. canteen staff, school principal) may report different barriers and facilitators to implementation of menu labeling. We will ensure that this forms part of the testing synthesis.

Key study information will include study title, name of the first author, year of publication, country of study, language, study type (qualitative, quantitative and mixed methods studies), intervention type (e.g. food standards, nutrient standards), type of implementation  (voluntary or mandatory participation), study setting (primary, post primary, academies etc.) and participant characteristics (canteen staff, head teacher, contracted catering suppliers, food service directors and managers, programme coordinators etc.), sample size, data collection and analysis methods. Data on intervention effects/outcomes, such as change in children’s dietary habits will not form part of this review. The outcome that forms part of PICOS are the barriers and facilitators to the implementation of nutrition standards for school meals.

Moreover, a sensitivity analysis will be conducted in order to determine if the synthesis is sensitive to the following: study design, quality assessment, intervention type (e.g. food standards, nutrient standards), type of implementation (voluntary or mandatory participation), study setting (primary, post primary, academies etc.) and participant characteristics (canteen staff, head teacher, contracted catering suppliers, food service directors and managers, programme coordinators etc.) and location (e.g. Europe, America, Australia). As barriers and facilitators to implementation are highly context-dependent, the sensitivity analysis will be an important step in assessing if the synthesis is sensitive to contextual factors such as intervention type, implementation type and time of data collection.

Comment 10

While there is a clear overview/outcome of data extraction for the qualitative component I am unclear about the outcome for the quantitative aspect, is it an effect on nutrition, children’s diets, etc. A bit more clarification for the quantitative outcome is required.

Response 10

For quantitative studies the proportion of respondents (study participants) that identified each barrier/facilitator will be extracted, coded and mapped against the Theoretical Domains Framework. Study participants will be listed so as to be able to discern between the various supply side stakeholders. To provide some context on the extent of the barrier/facilitator, the frequency of each factor reported within studies will also be calculated. This information will be added to the manuscript.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Availability Statement

    Underlying data

    No underlying data are associated with this article.

    Extended data

    Open Science Framework: Barriers and facilitators to the implementation of nutrition standards for school food: a mixed-methods systematic review, https://doi.org/10.17605/OSF.IO/6Q24P 50.

    This project contains the following extended data:

    • -

      Supplementary File 2. Inclusion and Exclusion Criteria.pdf

    Reporting guidelines

    Open Science Framework: PRISMA-P checklist for ‘Barriers and facilitators to the implementation of nutrition standards for school food: a mixed-methods systematic review’, https://doi.org/10.17605/OSF.IO/6Q24P 50.

    Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CCO 1.0 Public domain dedication).

    Articles from HRB Open Research are provided here courtesy of Health Research Board Ireland

    RESOURCES