Framework for Selecting Best Practices in Public Health: A Systematic Literature Review

Abstract

Evidence-based public health has commonly relied on findings from empirical studies, or research-based evidence. However, this paper advocates that practice-based evidence derived from programmes implemented in real-life settings is likely to be a more suitable source of evidence for inspiring and guiding public health programmes. Selection of best practices from the array of implemented programmes is one way of generating such practice-based evidence. Yet the lack of consensus on the definition and criteria for practice-based evidence and best practices has limited their application in public health so far. To address the gap in literature on practice-based evidence, this paper hence proposes measures of success for public health interventions by developing an evaluation framework for selection of best practices. The proposed framework was synthesised from a systematic literature review of peer-reviewed and grey literature on existing evaluation frameworks for public health programmes as well as processes employed by health-related organisations when selecting best practices. A best practice is firstly defined as an intervention that has shown evidence of effectiveness in a particular setting and is likely to be replicable to other situations. Regardless of the area of public health, interventions should be evaluated by their context, process and outcomes. A best practice should hence meet most, if not all, of eight identified evaluation criteria: relevance, community participation, stakeholder collaboration, ethical soundness, replicability, effectiveness, efficiency and sustainability. Ultimately, a standardised framework for selection of best practices will improve the usefulness and credibility of practice-based evidence in informing evidence-based public health interventions.

Significance for public health.

Best practices are a valuable source of practice-based evidence on effective public health interventions implemented in real-life settings. Yet, despite the frequent branding of interventions as best practices or good practices, there is no consensus on the definition and desirable characteristics of such best practices. Hence, this is likely to be the first systematic review on the topic of best practices in public health. Having a single widely accepted framework for selecting best practices will ensure that the selection processes by different agencies are fair and comparable, as well as enable public health workers to better appreciate and adopt best practices in different settings. Ultimately, standardisation will improve the credibility and usefulness of practice-based evidence to that of research-based evidence.

Introduction

Practice-based evidence in public health

According to the World Health Organization (WHO), public health is defined as all organised measures (whether public or private) to prevent disease, promote health, and prolong life among the population as a whole.¹ Public health activities can be generally categorised into five areas, namely monitoring and evaluation, health promotion and protection, healthcare service delivery, health system as well as research (Appendix 1).^2-4

Evidence may be defined as the available body of facts or information indicating whether a belief or proposition is true or valid.⁵ Similar to clinical medicine,^6,7 evidence-based public health emphasises proof of efficacy,^8,9 so that scarce resources are efficiently utilised on interventions which have been shown to bring about desired outcomes,¹⁰ and that benefits outweigh harm for both individuals and society.¹¹ Furthermore, it encourages accountability by decision-makers through the use of objective judging metrics to evaluate evidence.¹²

Many sources of evidence may be considered in decision-making. However, the quality of evidence is often only judged by the internal validity of study designs,¹³ with randomised controlled trials held as gold standard for minimising bias in study results. This then limits the data considered to research-based evidence in many cases. For example, the WHO Guidelines Review Committee evaluates evidence using the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach,¹⁴ where one of the assessment criteria for high confidence in the body of evidence is low methodological bias.¹⁵ While efficacy under controlled conditions supports causality between the intervention and outcomes, there is growing awareness of the importance of demonstrating effectiveness in actual programme settings, on top of efficacy, for public health interventions.^16-18 Evaluating a programme in the real world not only considers interactions between contextual factors and the intervention to ensure feasibility, or external validity, but also broadens the scope of interventions that can be assessed beyond simple individual-based interventions favoured by randomised controlled trials.^19,20 Contextual factors such as social determinants of health,¹⁰ the ability of public health workers to deliver the interventions to target groups,¹⁹ and accessible resources²¹ all alter effectiveness of interventions in reality. While pragmatic trials may be an alternative in that they are carried out under real-life conditions while preserving internal validity,²² population-wide upstream interventions, for instance policies to promote healthy diets in Finland, remain incompatible with trial settings²³ and require more qualitative methods of assessment.²⁴ Therefore, practice-based evidence, as opposed to research-based evidence, has been proposed as a more relevant source of evidence for public health decision-making due to the focus on populations as a unit and complexity of multi-disciplinary interventions.²⁵ Furthermore, use of practice-based evidence may enhance the translation of interventions from research to practice, a problem often cited in literature,²⁶ by considering drivers and barriers in implementation. In this review, practice-based evidence is obtained from field-based assessment of an intervention in a specific real-life setting.^27,28 In contrast, research-based evidence refers to empirical data derived from testing hypotheses about the efficacy of the intervention. This may be done through observational or experimental studies, with biases and confounders minimised to elucidate the relationship between the intervention and observed results.²⁹ This review proposes that both sources of evidence are complementary in informing evidence-based practice in public health and greater attention should be given to helping decision-makers tap available practice-based evidence.

Evaluation of existing public health interventions is one valuable source of practice-based evidence.¹¹ While frameworks have been developed to direct the process of evaluation,³⁰ relatively little work has been done to outline criteria for reviewing practices for evidence.³¹ In addition, programme evaluations are often geared towards ensuring accountability towards funders or improving the programme itself, rather than for sharing of lessons learnt.³²

Best practice approach

In particular, selection of best practices is one way via which implemented interventions may be evaluated to generate practice-based evidence.³³ The concept of systematically identifying best practices first started in the private sector,³⁴ where a best practice refers to a model of excellence against which counterparts in the industry can benchmark their own operations to better performance.^35,36

In the public sector, the notion of best practices also underlies policy transfers between countries³⁷ and is increasingly utilised in various sectors including education,³⁸ immigration³⁹ and public health, the focus of this review.A practice may be broadly taken to mean a policy, activity, intervention, approach, programme and so on.^40,41 Similar to industries, the objective of identifying best practices in public health is to avoid wasting resources on reinventing the wheel by learning from others under comparable circumstances.^34,42 Such exchange of knowledge not only facilitates improvement of current practices, but also helps those starting new interventions to avoid common mistakes⁴³ and accelerate programme development.⁴⁴ The increased collaboration and learning between organisations are also in line with the global movement to promote knowledge management as a means to improve outcomes.^45-47 Nonetheless, the best practice approach requires dedicated resources for programme evaluation and proper documentation.⁴⁸ The lack of consensus on the definition and criteria for best practices,^42,49 also impedes the use of such practice-based evidence.³³ In addition, the reliability and credibility of practice-based evidence depend on a flexible and transparent evaluation process,⁵⁰ which has been unexamined thus far. Hence, there is an urgent need for this novel review to address the gaps in current literature and facilitate optimal utilisation of valuable practice-based evidence in the form of best practices.

Aim of study

This systematic literature review aims to develop a scientifically sound and feasible framework for the selection of best practices in public health. This seeks to address the research question: what are suitable measures of success of a public health intervention to generate practice-based evidence? Although a review by Baker similarly sought to identify criteria for evaluating research-based and practice-based evidence, the consolidated practice-based criteria were solely derived from 12 expert interviews.³¹ Hence the comprehensiveness of identified criteria was highly dependent on the knowledge and experience of the 12 public health experts, unlike in a literature review which can cover more extensive and varied sources to incorporate a wider range of opinions. This paper is thus likely to be the first systematic literature review that attempts to synthesise criteria for producing practice-based evidence in public health to guide organisations and decision-makers in identifying and learning from best practices. Furthermore, the interviews were conducted in 2004 and this systematic review will provide an update to the criteria identified in Baker’s paper.

Methodology

Search strategy

Literature search (April to June 2014) was carried out via three main strategies. Literature found through preliminary searches was consulted to develop appropriate search terms.^31,51

Firstly, Pubmed (1966-June 2014) and the Global Health Library (2005-June 2014) databases were searched for evaluation frameworks relevant to public health. Secondly, websites and publications of major international health-related organisations were searched to identify criteria and methodologies that had previously been used to select best practices in public health. The WHO library database (WHOLIS, 1948-June 2014) and Intergovernmental Organisation search engine (IGO) were further utilised to ensure the comprehensiveness of the literature search. Lastly, the reference lists of identified articles were hand-searched to select appropriate sources. Details of the search strategy are summarised by database in Appendix 2.

Inclusion and exclusion criteria

In all cases, the search was limited to English records with full text available online (including library searches with available subscriptions at Imperial College London, University of Oxford and World Health Organization) because of practical considerations. Articles looking at best practices or evaluation outside the scope of public health were excluded. In addition, articles listing case studies as best practices without accompanying definition, criteria or selection methodology were also eliminated. However, there was no restriction on the type of literature and grey literature, such as websites and meeting reports, was included. The inclusion and exclusion criteria are summarised in Appendix 3.

Data extraction and synthesis

Once selected, data extracted included basic information like authors and year of publication, definitions of best practice if given, as well as methodology (e.g. expert panels, scoring system) and criteria used to evaluate public health interventions or select best practices.

Unlike standard systematic reviews for public health interventions,⁵² commonly used data quality assessment tools could not be applied due to the unconventional types of articles included and the focus of the study. Nonetheless, in line with the emphasis on using theories to enhance interventional effectiveness,^52,53 data were extracted on whether the theory supporting the evaluation frameworks or best practice selection methods was reported. This was then used to identify high quality papers. It also ensured that the criteria included in the final framework are aligned with public health principles and theories, and not skewed towards any organisation’s interests projected onto their criteria. In addition, evaluation or selection frameworks that were published in peer-reviewed articles were also considered to be of higher quality than other types of documentation as the former are likely to be more robust studies having been through the peer-review process.

Finally, qualitative data synthesis was performed. Various public health evaluation frameworks were first compared to identify common categories and criteria under each category. Frameworks which have been applied in diverse settings were considered more likely to be acceptable to public health workers. The categories derived from these evaluation frameworks then provided a structure for the proposed framework to ensure that all important aspects of a practice are assessed when generating practice-based evidence. Subsequently, criteria previously applied in the selection of best practices were compiled and classified according to the categories outlined. Once again, a criterion which was consistently used by different organisations is possibly widely accepted as a significant indicator of a successful public health intervention and feasible for application, and hence more likely to be included in the final framework.

Results

A total of 6889 records were obtained, of which 176 were eventually included in this literature review (Figure 1). The complete list of included records is provided in Appendix 4, sorted by database and order of extraction.

Figure 1.

Results of literature search.

Definition of best practice

To develop a framework for selecting best practices, a working definition of best practice is first necessary. One book, 10 peer-reviewed articles and 21 organisational sources included in this review provided varying interpretations. Alternative terms include good practice, effective solution, promising practice and innovative practice.^{31,40,47,54-58} Good and effective practice are often used to avoid debate about whether a single perfect intervention exists.⁴⁷ Nonetheless, the process of selecting best practices is comparable to that for good practices and the former is preferred in this review as it provides greater incentive for countries and organisations to improve their practices.⁵⁹ However, best practice should not be understood in its superlative form,⁴³ but instead seen to encompass interventions that meet a set of pre-defined criteria to varying degrees and reflect the society’s or organisation’s priorities over time.³³ In addition, best and good practices were occasionally differentiated from promising or innovative practices by the level of data supporting the success of the intervention. Best practices are well-established programmes proven to be effective through rigorous evaluations whereas promising or innovative practices are still in their infancy but show some signs of potential effectiveness in the long run.^40,55 While having robust evidence to back the success of the practice is ideal, it is perhaps more important to document the level of evidence available to guide decision-makers who are trying to learn from these practices instead of creating separate labels with distinct criteria. Differentiating practices by the level of evidence may also limit the settings and types of interventions that may be included. Therefore, best practices also encompass promising practices with varying levels of supporting evidence in this review.

Kahan, Goodstadt and Rajkumar summed up six frequently used versions of the best practice approach in health promotion, which is useful for clarifying the definition of best practice in this review.^50,60 In accordance with the aim of generating practice-based evidence, this review is limited to best practices that are selected against previously established criteria to illustrate what works in reality. This is preferred over using best practices as a compilation of enabling elements, standards or steps that are detached from any cultural or social context and hence difficult to apply in the real world.⁵⁹

The importance for best practices to demonstrate effectiveness or positive outcomes with regard to programme objectives in a specific real-life context is supported by 25 reviewed sources.^{17,41-43,47,48,54-56,58,61-75} Additionally, in order to fulfil their purpose as a learning tool across countries and organisations, best practices should be defined by their potential for adaptation to other settings through consideration of implementation and contextual factors, as agreed by 17 sources.^{17,41-43,47,49,55-57,64,68,69,71-75}

In short, a proposed working definition for best practices is: practices that have shown evidence of effectiveness in improving population health when implemented in a specific real-life setting and are likely to be replicable in other settings. Consequently, the emphasis on real-life implementation also requires evaluation with a focus on contextual and implementation factors as compared to experimental settings.⁶⁸

Existing frameworks for public health evaluation

With reference to the five areas of public health activities mentioned in the introduction (and elaborated in Appendix 1), only one peerreviewed article was focused on evaluating public health surveillance while 39 peer-reviewed articles and nine organisational sources offered diverse frameworks for evaluation of health protection and promotion practices. In addition, three peer-reviewed and two organisational sources focused on healthcare services delivery, and five peerreviewed and seven organisational sources explored health system evaluation. There were no records found for the evaluation of public health research. This is expected due to the search terms used in the literature search to elucidate criteria for the evaluation of practicebased evidence specifically, as opposed to research-based evidence which is beyond the focus of this review.

Regardless of the area of public health activities, the review found a general consensus on the importance of assessing the implementation process as well as short-term and long-term outcomes, as recommended by 36 sources.^{30,48,63,72,76-107} Monitoring the implementation process strengthens the causal relationship between the intervention and observed outcomes by providing information on the facilitating intermediary links.⁸⁹ In cases where outcome indicators are unavailable, process evaluation may also act as a provisional indicator of effectiveness in view of its expected impact on outcomes.⁹¹ For instance, monitoring the coverage of a target group for a screening programme may be used to gauge the success of the programme on top of outcome data on decreased morbidity and mortality, which may only improve after a long period and are difficult to attribute to a single intervention. On the other hand, monitoring outcomes of public health practices ensures that desired objectives are met and guides resource allocation in favour of effective practices.⁷⁶ Alternatively, the RE-AIM framework first proposed by Glasgow attempts to condense process and outcome evaluation into five dimensions, namely reach, efficacy,¹⁰⁸ adoption, implementation and maintenance. Outcome evaluation involves the measurement of the size of change in a specific desirable outcome (efficacy), while the implementation process is assessed through the percentage of target population receiving the intervention (reach), proportion and representativeness of settings taking up the intervention (adoption), degree to which the intervention is carried out as planned (implementation), as well as sustainability of the intervention and its effects (maintenance). Each of these domains then contributes to the public health impact of the intervention. This framework is an extension of Abram’s proposal that the impact of a programme can be equated to the product of its reach and efficacy.¹⁰⁹ Since its introduction in 1999, the RE-AIM framework has been widely used to assess various health promotion programmes in peer-reviewed studies.^110-120 Manipulating the five dimensions further produces additional criteria such as efficiency, given by (reach × efficacy)/cost of the intervention.¹¹⁰

Another category typically included in public health evaluations is the context. Contextual factors include programme inputs as well as characteristics of the health issue and target community. Assessment of contextual factors ensures that the intervention is relevant to the target group’s needs and circumstances.^48,121,122 Furthermore, it also pinpoints factors that may influence the ability of the programme to achieve its desired outcome,^{85,87,89,92-95,123} such as political environment and available resources. Since a best practice needs to be transferable between settings, consideration of the context during evaluation is useful in facilitating replication of the practice.¹²³ Hence, inclusion of context, process and outcome evaluation criteria is likely to be fundamental for a framework generating practice-based evidence in public health, which is in line with the theory of change and realistic approach to evaluations where analysis of background factors and process descriptions are necessary to explain outcomes in the complex system of public health.^123-125

With regard to the programme content, programmes may sometimes be appraised by their objectives, theoretical underpinnings and scope of interventions. Interventions based on theories, such as the health belief model for behavioural interventions, are likely to facilitate positive out-comes.^{30,63,78,84,86,121,122} A comprehensive intervention takling both individual and wider health determinants is also aligned with health promotion goals to build an environment that supports healthy lifestyles.^83,121

In addition, the quality of evidence illustrating the effectiveness of an intervention is occasionally included as part of the evaluation framework.^{58,88,126,127} Alternatively, programmes may simply be assessed by the presence of formal formative, process and/or outcome evaluation studies.^48,87

It is interesting to note that health system assessments also focus on sustainable and equitable financing of the health system,^128,129 an element rarely found in health promotion or healthcare service delivery evaluation. In some cases, fair financing may be subsumed under equity.^{101,103,104,128-131}

Lastly, some evaluations found do not allow for assessment of the programme against standards or comparisons between programmes^132-134 and are hence not useful for the selection of best practices. Some reviewed sources also provided individual points of evaluation, highlighting attributes of an ideal public health activity.^{30,48,58,80,86,87,93,94,126, 127,135-139} Appendix 5 summarises the various points of evaluation cited according to the five main categories described earlier, namely, context, content, process, outcomes and evaluation.

Previously used methodology for selecting best practices

Seven peer-reviewed articles and 32 organisational sources detailed the process by which best practices were selected or public health practices were evaluated. Understanding the methodology will be valuable for the development of a best practice framework as it affects how the framework will be applied and hence its design to enhance usability.

Subjectivity at various stages of selection or evaluation is a universal feature across all reviewed sources. Firstly, best practices were identified from submitted cases,^{41,43,47,54-56,59,66,68,75,88,140-152} or from literature review and experts’ opinion.^{44,60,61,64,153-156} For the former, programme managers were commonly required to submit a completed template, which was usually narrative and qualitative to accommodate the uniqueness of the practice.⁵⁹ The range of interventions identified and quality of information used for assessment are hence reliant on the initiative of the programme managers. In the case of evaluations by experts, the programme managers were instead contacted or reviewers visited intervention sites,^17,137 and the selection of interventions to be evaluated was limited by the biases and knowledge of the expert reviewers. In one exception, practices were shortlisted from projects related to United Nations agencies.¹⁵⁷ As each approach has its pros and cons, it is important for reviewers to report the limitations of their method of choice and any potential bias on the type of practices selected. Subsequently, reviewers appraise the information collected against pre-set criteria to determine if the intervention can be considered a best practice. Hence, the composition of the panel of reviewers and the selection process are also crucial in ensuring valid and reliable selection of best practices. While reviewers are typically branded as experts in the relevant fields, there is often little information on the background of the reviewers. Besides professional academics or organisational staff, it has also been suggested that best practices should be assessed by the beneficiaries of the intervention.^33,59 This adheres to the recommendations for participatory evaluation in public health and allows direct assessment of the acceptability of the intervention.³⁰ However, it may be difficult to engage the most economically disadvantaged beneficiaries of the intervention in reality and to ensure an equitable representation.³³ Regardless, careful consideration of the composition of the panel of reviewers is important to avoid biases due to vested interests and details of the composition should be made transparent.

Next, the selection of best practices or evaluation of public health interventions is typically done via consensus after independent assessment by each reviewer.^48,60,88 Alternatively, reviewers may be asked to score a programme on each criterion, which will then contribute to the overall score of the programme to facilitate comparisons with other programmes.^55,142,145 Either way, subjective views of each reviewer greatly influences the selection and evaluation outcomes and a reliable selection framework should thus consider such subjectivity through appropriate criteria and indicators.

A few studies attempted to provide more objective evaluation methods. Some users of the RE-AIM model have translated each criterion into a mathematical formula, for instance impact of an intervention can be calculated as (reach × efficacy).¹¹⁷ Similarly, Reed proposed a scoring system given by (impact of the intervention × number of people implementing the intervention other than programme staff or percentage of target population involved).¹⁵⁸ While these methods may facilitate comparison between different interventions based on their scores, they may also oversimplify or ignore important qualitative information. Furthermore, some degree of subjective judgements is unavoidable in any evaluation, for instance in weighing the importance of the various criteria used.¹¹⁷ Therefore, it is more important to acknowledge the subjectivity in the evaluation process and incorporate it into the proposed framework for evaluating best practices rather than trying (and failing) to eliminate it fully.

Two sources found by this review also recommended validating best practices through rigorous empirical studies of the causality between intervention and outcomes.^40,159 Once again, best practices should not only be sourced from settings conducive for research but also other real-life settings where process evaluation may then be used to support any observed correlation between the programme and outcomes in replacement of empirical causality studies.

Nine reviewed sources further provided considerations for choosing indicators in public health evaluation.^{76,97,99-101,106,139,160,161} Criteria set the benchmark for evaluating an intervention while indicators are measurements used to assess achievement of these standards.⁷⁶ Ideal indicators are valid because they provide an accurate reflection of what is being measured, relevant because they measure an important phenomenon, practical and cost-effective given accessible information, sensitive to changes arising from programme implementation, reliable in producing consistent results independent of reviewer or time, produce comparable measurements when used in different contexts, easy to interpret and useful for the purpose of selecting best practices.

Criteria used to select best practices

One book, eight peer-reviewed articles and 39 organisational sources presented various criteria for selecting best practices. To ensure that the criteria provide a comprehensive assessment of public health interventions, the criteria were grouped according to the five categories typically used in public health evaluations mentioned earlier, namely context, content, process, outcomes and evaluation. Appendix 6a-e lists the criteria that are cited by each source, with sources arranged by types and databases. Appendix 6a cites peerreviewed articles and book while Appendix 6b cites organisational sources from the database of WHOLIS and WHO Regional Offices of Africa and the Americas. Similarly, Appendix 6c includes sources from WHO Regional Office for Europe, UNDP and the World Bank; Appendix 6d cites sources from US CDC, IGO search engine and hand-searches of reference lists; and Appendix 6e focuses on sources derived from hand-searching reference lists. Appendix 7 then summarises the number of sources citing each criterion.

Almost all sources (44 of 48 sources) agreed that a best practice must be effective and show measurable positive results in achieving pre-defined objectives. This also supports the working definition of a best practice stated earlier, where a best practice must demonstrate what works in reality. Assessing effectiveness is likely to be crucial regardless of the area of public health, as earlier demonstrated. Additional sources providing detailed analysis of each individual criterion were also found. On top of the 48 sources, three additional peerreviewed sources offered different ways of evaluating effectiveness. Lengeler assessed individual and community effectiveness,¹⁶² while McDonnell proposed that effectiveness can be measured as (programme efficacy × probability that the programme can deliver its intended outcomes), where the latter is dependent on available human resources, infrastructure and the community’s access to the programme.¹⁶³ Lastly, Macdonald argued for inclusion of qualitative process indicators in addition to the final outcome evaluation.¹⁶⁴ While Macdonald’s claim is supported by evaluation frameworks described earlier,^{30,48,63,72,76-107} Lengeler’s and McDonnell’s work are less relevant to this review due to the emphasis on scientific studies to establish individual effectiveness and programme efficacy respectively, which may not always be feasible. In short, all positive and negative outcomes of an intervention across time should be taken into account when assessing effectiveness. The objectives of the intervention may also suggest potential targets, for instance achievement of more than 90% of the objectives.¹⁴⁵

Another commonly-cited criterion is programme sustainability (32 of 48 sources). On top of the 48 sources, eleven additional peer-reviewed articles provided definitions of sustainability in public health practices. Sustainability may be seen as a long-term continuation of i) activities through local ownership or incorporation into standard practices, otherwise known as institutionalisation;¹⁶⁵ ii) benefits as outlined in the objectives of the intervention, including health improvements or heightened attention on the issue; iii) community or organisational capacity to deliver the intervention; or iv) a combination of all three dimensions.^5,166-170 In particular, long-term availability of necessary resources, financial or otherwise, to run the intervention is an important point to consider as it greatly affects the maintenance of the activities and their benefits.¹⁷⁰ This is especially crucial for externally supported programmes as termination of the programme with the end of funding may limit the potential benefits that can be reaped from initial investments and erode the trust that the community placed in public health workers.⁵ Contextual and programme elements which may enhance sustainability include alignment of the intervention with national goals, political commitment, community participation, stakeholder partnerships and programme evaluation.^171-174 In addition, the timing of appraisal is also crucial, with some sources suggesting that a sustainable intervention should continue for a minimum of five¹⁷⁵ or two years after its start^54,141,149 or at least one year after the external funding stops.¹⁷⁰ Although Stephenson attempted to provide indices to measure sustainability,⁵ there is no agreed threshold for categorising a practice as sustainable or otherwise.¹⁷⁰ Hence, when selecting best practices, it is also important to state the duration of implementation prior to evaluation to inform decision-makers, as well as recognise that sustainability is a continuum where programme elements, benefits and community capacity are maintained to different extents.

Efficiency (24 of 48 sources), or cost-effectiveness, is important in ensuring that scarce resources are used in a prudent and accountable manner, and may be commonly defined as the ability to produce optimal results with minimum resources.^43,54,154 On top of the 48 sources, four additional peer-reviewed articles gave further examples of assessment of efficiency in practice. Where the cost of the intervention is known, cost-effectiveness analysis is commonly done^176,177 and a threshold can then be applied to determine if the intervention is cost-effective,¹⁷⁸ for example by relating the cost per Disability-Adjusted Life-Year averted to the per capita Gross Domestic Product of the country.¹⁷⁹ However, in absence of cost-effectiveness calculations, judging the efficiency of practices may rely on evidence of wastage avoidance, cost minimisation^56,62 or optimal use of locally accessible resources.¹⁸⁰

Potential for replication, or replicability, should also be a main criterion for defining best practices in line with the working definition (24 of 48 sources). Replicability may be defined as the ability of the intervention to continue achieving desired outcomes when adapted to various cultures and settings.^54-56,70 Thus, best practices should have key success factors that are independent of the context and available resources.

Next, relevance of the intervention in addressing an important public health issue in the community (21 of 48 sources) depends on the priorities and perceptions of the target community.^{43,154,155,175} This requires analysis of the disease burden and community profile as well as a needs assessment involving the target community before designing the intervention. Contextual factors such as integration of the intervention into existing structures and its culture appropriateness may also be considered under relevance.^70,155 Thus, awareness of the community and settings enhances relevance of the intervention. Furthermore, these descriptions may contribute to the replicability of the intervention by providing information on any contextual factors which may have led to the outcomes observed.

Stakeholder collaboration (17 of 48 sources) and community participation (12 of 48 sources) are two frequently used criteria for selecting best practices and in process evaluation in public health. Both elements are recommended to enhance local ownership,^47,54,62 increase the reach of the intervention,¹⁴⁸ capitalise on various competencies and incorporate perspectives of the target beneficiaries. Achieving these elements is thus believed to augment effectiveness and sustainability.^61,78,181 Furthermore, they are aligned with the principles of public health of being participatory and multi-sectoral in recognition of the fact that improvement in population health requires the collaborative efforts of more than any single actor.^4,42,121 On top of the 48 sources, five peer-reviewed articles on community engagement^182-186 and one on stakeholder collaboration¹⁸⁷ further contribute to this discussion. Markers of ideal community participation include appropriate members and participation process, empowerment of the individuals involved, improved community ties, synergistic and viable coalition where new ideas emerge as well as effective leadership and management. Thus, capacity building of the community (eight of 48 sources) may also be subsumed under community participation as a means through which the latter promotes effectiveness and sustainability of the intervention. On top of the 48 sources, six peer-reviewed articles on capacity building outlined the areas in which empowerment may be observed.^188-193 Development of the community’s knowledge and skills on health issues and ways to tackle them, community networks, leadership, resource mobilisation, investments, and organisational structures to support delivery of public health activities will facilitate the maintenance of the intervention and its benefits over time and even aid the community in managing other public health concerns in the future. On the other hand, stakeholder collaboration can be gauged by the synergy achieved, where comprehensiveness and innovation are enhanced by the pooling together of complementary resources and competencies.¹⁸⁷ Therefore, community participation and stakeholder collaboration are criteria that both support the attainment of positive outcomes and are themselves important public health goals to be achieved.⁸⁰ They are also assessed through proxy outcomes such as community empowerment and synergistic cooperation.

Ethical soundness of an intervention (14 of 48 sources) includes respect for an individual’s rights and dignity as well as professionalism by public health workers.^41,43,56,154 Reflecting fundamental public health principles, ethical considerations should underpin all activities involving human participants and be made explicit as a criterion for best practices. Furthermore, ethical interventions that do not infringe on an individual’s rights and a community’s norms are more likely to be accepted and utilised by the target group,⁵⁴ thus enhancing impact through greater reach and adoption.¹²⁶ Ethical frameworks proposed by eight peer-reviewed articles on top of the 48 sources are also considered. Suggested elements include prevention of harm while ensuring benefits at both individual and population levels, consideration of equity in distribution of benefits and burdens (two of 48 sources), respect for an individual’s autonomy and privacy, informed consent, consciousness of local norms, accountability as well as awareness of vulnerable groups^51,194-200 (eight of 48 sources). Specifically, equity may be assessed by the distribution of access, financing and effects^201-203 across place of residence, race, occupation, gender, religion, education, socioeconomic status and social capital.²⁰⁴ Davies further suggested eight elements that promote equity, including activities addressing social determinants of health.²⁰⁵ In short, ethical soundness is a basic requirement for any public health intervention and is taken to include equity and social inclusion of vulnerable groups in this review.

Having a theoretical basis underlying public health programmes (11 of 48 sources) and the need for strong evidence of effectiveness (12 of 48 sources) have been mentioned earlier under the chapter on existing frameworks for public health evaluation. While having a theoretical basis will be ideal to explain the logic behind public health interventions, an intervention should not be discounted as ineffective solely because of a lack of underlying theory as inclusion of contextual and process criteria may already provide information about the mechanisms leading to the observed outcomes.⁸² Similarly, the need for high quality evidence may restrict the selection of best practices to only those amenable to empirical studies and with ample resources to conduct these studies. This may again result in the drawbacks of using only research-based evidence. Therefore, in order to maximise learning from successful initiatives without a theoretical basis or rigorous evaluation, they should still be included in the selection for best practices, though with the level of evidence explicitly stated to inform decision-making.

The next criterion to be discussed is the innovative nature of interventions (11 of 48 sources), defined as those implemented in the context for the first time.^{54,141,148,150} However, an effective intervention should not be ruled out as a best practice simply because it has been applied previously and thus innovation will not be a significant criterion for selecting best practices. While outcome-related criteria such as effectiveness and sustainability are cited at similar frequency in peer-reviewed and organisational sources, it should be noted that the two types of sources differ in their emphasis on other categories of criteria.

In addition, while most peer-reviewed articles included the need for robust supporting empirical evidence and theoretical backing, criteria such as innovation and ethical soundness are rarely mentioned in these articles but frequently used by organisational sources. This supports the need to include views of both public health researchers and field workers to ensure a comprehensive review through consideration of both peer-reviewed and organisational sources.

Lastly, whether a programme is implemented as planned or well-executed (11 of 48 sources) depends on the required resources and reflects the feasibility of the programme in the context of the target group.²⁰⁶ Other less commonly used criteria also include extensive reach or scale of intervention (nine of 48 sources), which may penalise small-scale targeted initiatives; mandatory formal evaluation studies (seven of 48 sources), which may be restrictive given availability of resources in different settings; support from leaders (seven of 48 sources), which can conceptualised as expressed commitment in public statements, building of relevant infrastructure or budget allocation;²⁰⁷ having clearly defined objectives (six of 48 sources); comprehensiveness (six of 48 sources), such as targeting both determinants of health and environmental factors; integration into local context (five of 48 sources); acceptability (two of 48 sources) and visibility (one of 48 sources). As these criteria were found to be less commonly cited, used without accompanying justification or intermediary towards effectiveness and sustainability of an intervention, they are not considered as essential features of a best practice in this review. Nonetheless, ensuring that these less commonly cited factors are met may further enhance the implementation and effectiveness of the interventions, for instance gaining the buy-in and support from community leaders to ensure the intervention is sustained over time.

Proposed evaluation framework for selection of best practices

Based on the literature review of past evaluations of public health interventions and selections of best practices, this paper proposes a framework for selection of best practices in public health as illustrated in Table 1. Contextual and process elements should be considered together with the outcomes of a practice, as they can further direct adaptation to other settings. Eight criteria across context, process and outcomes were chosen based on their widespread application in evaluation frameworks for various areas of public health as well as best practice selection processes: relevance, community participation, stakeholder collaboration, ethical soundness, replicability, effectiveness, efficiency and sustainability. While the three criteria of having a theoretical basis underlying the intervention, showing strong evidence for effectiveness and being innovative were cited by a number of sources, they were excluded from the final framework as they are not critical for identifying a best practice worthy of emulation as discussed earlier.

Table 1.

Proposed framework for selection of best practices in public health, with examples for each criterion.

Category Criterion		Example
Context 1.	Relevant Relevant to the needs of the community (conduct problem analysis and needs assessment of the community prior to programme development; consider perspectives of the target group and stakeholders) Relevant to the setting of the community(describe characteristics of the community and context)	Evaluate disease burden in community. Involve target groups and stakeholders in needs assessment. Describe existing programmes, social and cultural perception of disease.
Process 2.	Engage the community (community participation) Describe who and how members of the community are involved Empower the community Achieve synergy through community participation in programme development and implementation	Ensure appropriate representation of target groups, including vulnerable groups. Improve knowledge about the disease in local community. Give rise to new approaches due to inclusion of the community.
3.	Involve the right stakeholders (stakeholder collaboration) Ensure appropriate representation of relevant stakeholders Describe who and how stakeholders are involved Achieve synergy through stakeholder collaboration	Give rise to new approaches due to pooling together of resources and competenc. of non-governmental organisations, donors and international organisations.
4.	Ethically sound Ensure benefits outweigh harm to individuals and community Distribute access, financing, benefits and harms equitably Demonstrate respect for individual autonomy and privacy Consider vulnerable groups Ensure accountability to community Demonstrate respect for local norms and cultures	Promote fair distribution of benefits across ethnicities, socioeconomic status and gender. Involve voluntary participants. Target the poor and females. Benefit the local community and do not deplete local resources.
5.	Replicable* Require expertise and resources that are generalisable to other settings
Outcomes 6.	Effective* Achieve desirable outcomes and improve public health Describe types of supporting evidence available	Reduce morbidity and mortality, achieve universal access to healthcare, and enhance community awareness. Conduct case control studies, patient surveys, and routing monitoring.
7.	Efficient Describe physical, financial and technical resources used Use locally accessible resources Demonstrate minimisation of resource use and wastage Describe types of supporting evidence available	Describe expertise of public health workers required, and costs of intervention. Conduct cost-benefit analysis, and cost-effectiveness analysis.
8.	Sustainable Demonstrate (potential of) continuation of programme activities through local ownership or institutionalisation Demonstrate (potential of) continuation of benefits of programme Demonstrate (potential of) continuation of community and organisational capacity to deliver programme, including source of funding in the long run State duration of programme since start of implementation	Train local public health workers to administer intervention. Ensure community awareness continues after programme ends. Self-financing of intervention by community.

*Replicability and effectiveness are fundamental criteria in line with the working definition of best practices

Keeping in mind the subjective nature of the selection process, the background of reviewers and process of selection (whether the practice is identified from submitted case studies or by reviewers themselves) should be made transparent to inform decision-makers of the potential biases. Sub-points and examples are also developed for each of the eight criteria in the proposed framework to guide reviewers. Indicators specific to the health issue of interest may also be chosen to provide additional measurements to facilitate selection. By structuring the framework as a checklist, it not only guides the selection process but also facilitates reporting and dissemination by identifying strengths and weaknesses of the practice of interest. It is important to note that besides effectiveness and replicability, best practices may not necessarily exhibit all the listed criteria.⁷¹ However, they should not go against any of the listed points. For instance, while best practices may not have demonstrated a causal link with a decrease in mortality, they should not show an increase in mortality instead. To ensure replicability of best practices in other settings, descriptions of the community, context, resources employed as well as supporting evidence should ideally be included to guide decision-makers in adopting these best practices to their own context. Ultimately, in order for the best practice approach to achieve its purpose as a learning tool, the context-specific nature of programmes must be stressed and elements should be adapted before replication in other settings.³³

Lastly, the proposed framework is deliberately kept general to be applicable to any public health action at all levels of implementation, be it individual-, community- or population-based interventions. Depending on the public health issue and types of interventions, the framework may be further fine-tuned to emphasise specific criteria.⁸⁶ For instance, when applying the framework to health systems, equity and sustainability of the health financing mechanisms may be given greater weight due to their importance.¹²⁹ The proposed framework should also be updated over time to reflect new priorities and focus of the society and organisation of interest.³³

Conclusions

While metrics for assessing empirical studies for the quality of evidence have been extensively examined, the lack of consensus on what constitutes practice-based evidence impedes utilisation of valuable real-world experiences in informing public health interventions. Best practices in public health may be defined as interventions that have been shown to produce desirable outcomes in improving health in real-life settings and are suitable for adaptation by other communities. By consolidating previous best practice selection criteria in secondary literature and comparing them with established public health evaluation frameworks, a framework covering eight criteria (relevance, community participation, stakeholder collaboration, ethical soundness, replicability, effectiveness, efficiency and sustainability) across programme context, process and outcome is proposed in this review to guide the selection of best practices.

Strengths and limitations of study

This review addresses a notable gap in current literature on standards for practice-based evidence in public health by proposing the first framework (Table 1) for the selection of best practices based on an extensive systematic review. Despite the increasing awareness about the benefits of practice-based evidence, there is a lack of consensus on the criteria for assessing such evidence, hence hindering its usability.³¹ Therefore, the framework suggested in this review provides a much-needed tool for public health workers to tap experiences in the field and evaluate interventions in a logical manner to select noteworthy practices which can then be adapted to other settings as evidence-based practices. Promotion of practice-based evidence widens the scope of evidence beyond research with regard to complex and multi-disciplinary public health work and this novel review is hence a crucial first step in setting standards for generating such evidence.

All literature included in the review was found using the methodology recommended by the Cochrane Collaboration for systematic reviews,²⁰⁸ from formulation of a research question to systematic search and selection of records. As the first systematic review to focus on elucidating criteria for practice-based evidence, this review is therefore a novel attempt to define and conceptualise elements of a best practice in public health in a scientifically sound manner. Furthermore, the use of broad search terms and the search through 17 databases with different focuses and settings aimed to minimise omission of crucial material and increase comprehensiveness and representativeness of included records.

However, the proposed framework is produced following literature search of the stated databases and is by no means exhaustive or conclusive. Due to time and resource constraints, the search was limited to records in English and articles with full text available online. Therefore, inclusion of more databases and removal of search restrictions may improve the comprehensiveness of this review. In addition, data search, extraction and synthesis were only conducted by a single reviewer. In order to minimise errors, search of and data extraction from the first database, Pubmed, were conducted twice and results compared to identify any oversight. Nonetheless, having a second independent reviewer would be ideal to prevent individual bias in the literature search and extraction. Due to the heterogeneity and nature of the literature included, there is also no available quality assessment tool that can be used to determine the bias of included records and this may affect the quality of this review. While extracting the theoretical basis for evaluation or selection and distinguishing between peer-reviewed and organisational sources may hint at the quality of the records, it may be useful to update this review when an appropriate quality assessment tool becomes available. Furthermore, as this is the first literature review that attempts to elucidate criteria for evaluating practice-based evidence, details of the methodology, including search terms and databases used, and the results obtained could not be corroborated with any existing study. Lastly, pilot testing of this framework in evaluation of existing programmes, for instance to evaluate health-related policies across different contexts in the recent movement to promote Health in All Policies in Europe,²⁰⁹ or gathering expert opinion will also be necessary in the future to determine its usability.

Future research

Future studies on improving best practice reporting, dissemination and adoption will be complementary in maximising the potential of the best practice approach.⁴⁸ Appropriate dissemination is likely to improve the efficiency of programme development as resources may then be devoted to adoption rather than innovation.⁷³ Suitable research may hence go beyond the boundaries of public health and involve knowledge management theories.^46,210 This includes building suitable knowledge sharing platforms and helping decision-makers identify elements that can be replicated to their own settings.

Ultimately, improving the selection, dissemination and transfer processes of best practices can then facilitate and promote appropriate use of the best practice approach to generate reliable practice-based evidence which can complement research findings in public health. Inclusion of credible practice-based evidence in informing evidence-based practice is consequently likely to enhance the feasibility of derived interventions and widen the scope of recommended practices. Furthermore, it taps a previously underrated wealth of field-based knowledge and experience that should be equally, if not more, valued as physical or financial resources for practitioners attempting to develop effective public health interventions.