Evidence on Scaling in Health and Social Care: An Umbrella Review

Abstract

Policy Points.

• More rigorous methodologies and systematic approaches should be encouraged in the science of scaling. This will help researchers better determine the effectiveness of scaling, guide stakeholders in the scaling process, and ultimately increase the impacts of health innovations.

• The practice and the science of scaling need to expand worldwide to address complex health conditions such as noncommunicable and chronic diseases.

• Although most of the scaling experiences described in the literature are occurring in the Global South, most of the authors publishing on it are based in the Global North. As the science of scaling spreads across the world with the aim of reducing health inequities, it is also essential to address the power imbalance in how we do scaling research globally.

Context

Scaling of effective innovations in health and social care is essential to increase their impact. We aimed to synthesize the evidence base on scaling and identify current knowledge gaps.

Methods

We conducted an umbrella review according to the Joanna Briggs Institute Reviewers’ Manual. We included any type of review that 1) focused on scaling, 2) covered health or social care, and 3) presented a methods section. We searched MEDLINE (Ovid), Embase, PsycINFO (Ovid), CINAHL (EBSCO), Web of Science, The Cochrane Library, Sociological Abstracts (ProQuest), Academic Search Premier (EBSCO), and ProQuest Dissertations & Theses Global from their inception to August 6, 2020. We searched the gray literature using, e.g., Google and WHO‐ExpandNet. We assessed methodological quality with AMSTAR2. Paired reviewers independently selected and extracted eligible reviews and assessed study quality. A narrative synthesis was performed.

Findings

Of 24,269 records, 137 unique reviews were included. The quality of the 58 systematic reviews was critically low (n = 42). The most frequent review type was systematic review (n = 58). Most reported on scaling in low‐ and middle‐income countries (n = 59), whereas most first authors were from high‐income countries (n = 114). Most reviews concerned infectious diseases (n = 36) or maternal–child health (n = 28). They mainly focused on interventions (n = 37), barriers and facilitators (n = 29), frameworks (n = 24), scalability (n = 24), and costs (n = 14). The WHO/ExpandNet scaling definition was the definition most frequently used (n = 26). Domains most reported as influencing scaling success were building scaling infrastructure (e.g., creating new service sites) and human resources (e.g., training community health care providers).

Conclusions

The evidence base on scaling is evolving rapidly as reflected by publication trends, the range of focus areas, and diversity of scaling definitions. Our study highlights knowledge gaps around methodology and research infrastructures to facilitate equitable North–South research relationships. Common efforts are needed to ensure scaling expands the impacts of health and social innovations to broader populations.

The scaling of health and social care innovations is a topic of growing importance in the contemporary world because of its potential to reduce waste and inequities in health and social settings and to expand the benefits of effective innovations. ¹ , ² , ³ , ⁴ , ⁵ , ⁶ The emerging science of scaling seeks to provide systematic, critical, principles‐based knowledge to inform and improve the scaling of health and social care. ⁷ , ⁸ Since 2010, a framework proposed by ExpandNet, a World Health Organization (WHO) initiative, has been widely consulted to guide scaling of evidence‐based health care innovations. It characterizes “scaling” practices as those whose goal is to increase the reach and adoption of innovations. ² In the literature, “scaling” is often interpreted as identical to “spread.” However, they present nuances of method and purpose. Although “spread” usually means replicating an innovation somewhere else, ⁹ the word “scaling” can also refer to the various methods used and steps required to increase the impact of an innovation, ⁷ i.e., the processes used to expand its reach, improve its quality, or increase access for vulnerable groups. In addition, “vertical” scaling describes the process of using macro structures (such as policy, regulation, or financial tools) to institutionalize an innovation throughout a whole system, whereas “horizontal” scaling refers to the rolling out of an innovation from one specific setting (e.g., a pilot project) to others. ² , ⁵ , ¹⁰ Furthermore, the term “scaling” is often suffixed with “up,” “out,” or “deep.” Briefly, “scaling up” means increasing throughput to more individuals (e.g., offering the same course to more health professionals), “scaling out” refers to expanding the innovation by adapting it to new systems or populations (e.g., offering the same course on an online platform), ¹¹ and “scaling deep” corresponds to improving the outcomes of the innovation by improving the structures and processes used (e.g., updating courses from an equality, diversity, and inclusion perspective). ⁷ Inspired by these definitions, ² , ⁵ , ⁷ for the purposes of this paper we define “scaling of health and social care innovations” as a systematic evidence‐informed process whose aim is to increase the positive impacts of an innovation that has proven effective for improving quality in care and the welfare of individuals and populations. Scaling can focus on any health or social outcome, e.g., the functionality of a health system (access to essential services, quality of care, the demand for essential services, or its resilience), medication access for tuberculosis infections, COVID‐19 screening, or treatment of depression. It can also focus on food security, empowerment of vulnerable communities, poverty, education, or human development. ¹² , ¹³ , ¹⁴ We define “innovations” as activities that are new as well as those that have already been successfully implemented in specific settings before being scaled to further contexts. ¹⁵ In general, they are perceived as new by end users. ¹⁶

There is considerable interest in ensuring that new knowledge born out of research is put to good use to improve population health. ¹⁷ , ¹⁸ , ¹⁹ The fields of knowledge translation (KT) and implementation science have improved the prospects of putting research into practice. Within these more established fields, scaling science is considered complementary. ⁷ , ⁸ , ²⁰ Most KT and implementation models such as the Knowledge‐to‐Action model ²¹ do not explicitly include scaling as an important and distinctive phase of the KT process. ⁸ We believe greater attention to the science of scaling can improve the impact of implementation and KT efforts.

The COVID‐19 pandemic increased the urgency of scaling health and social care innovations in high‐income countries (HICs) as well as in lower‐ and middle‐income countries (LMICs). ¹⁹ , ²² , ²³ , ²⁴ For example, the pandemic hastened the scaling of a UNICEF oxygen therapy project to over 90 countries, ²³ and in Australia, findings from a study of COVID‐19 transmission in schools and childcare settings in the early stages of the pandemic informed widely applied policies to reduce transmission in these settings. ²⁴ In addition, many policymakers believe that scaling effective health and social care innovations will reduce per capita cost of health care through economies of scale. ²⁵ , ²⁶ Moreover, policymakers who are concerned about the rapidly increasing inequities within and among countries consider scaling as an effective means to address political and community health inequities. ²⁷ , ²⁸ , ²⁹ , ³⁰ These pressing motivations to scale effective innovations are now giving rise to a new interest in the underlying mechanisms at play in scaling, and how to harness them. These mechanisms influencing an action or a system, through which an intervention works or fails to work, can be practical (e.g., human resources), social (e.g., cultural mores), or psychological (e.g., reactions or reasonings), all of which can vary depending on the area targeted for scaling. ³¹ , ³²

There is as of yet less evidence about scaling mechanisms or processes in health and social care than in business and agriculture. Indeed, there is evidence of a persistent failure to scale effective health innovations across health care systems around the world. ⁷ , ³³ , ³⁴ This may, in part, be attributable to a lack of scientific knowledge about scaling as well as a failure to share practices across countries. ⁷ , ³⁴ , ³⁵ The use of reviews of research evidence, and in particular, systematic reviews, is recommended to guide health policy, to support efforts to scale health and social innovations, and to identify gaps in the review evidence base. Previous studies have suggested that although systematic reviews on scaling in health and social care are increasing, they are often highly focused on a specific topic or clinical domain (e.g., HIV testing uptake) and provide incomplete and haphazard coverage of the evidence needs of stakeholders. ³⁴ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰

In this umbrella review, we therefore aimed to synthesize the evidence base for scaling in health and social care and to identify current knowledge gaps. The results will help to build and advance the emerging science of scaling.

Methods

Study Design

We conducted an umbrella review whose purpose was to synthesize the findings of available reviews. We adopted the Joanna Briggs Institute (JBI) methodology for umbrella reviews, ⁴¹ and we used the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) 2020 guidelines to structure this review (Additional File 1). ⁴² The study protocol was registered with the International Prospective Register of Systematic Reviews (CRD42020183774) on November 11, 2020. The present paper is the first iteration of a living umbrella review for which a protocol has been published. ⁴³

Eligibility Criteria

We collected all types of evidence matching the “PICO” criteria (Participants, Intervention, Comparator, Outcome) and “PICo” (Population, phenomena of Interest and Context) to capture the evidence from quantitative and qualitative reviews pertaining to scaling. For example, in addition to systematic reviews, we included critical reviews, scoping reviews, and narrative reviews.

Participants or Population

We considered all reviews that included primary studies focusing on individuals, health systems, services, or organizations that had been exposed to the scaling of a health or social care innovation.

There were no restrictions based on sociodemographic factors (e.g., age, ethnicity, socioeconomic status) or health conditions (e.g., comorbidities).

Interventions/Phenomena of Interest

We considered all types of interventions. This umbrella review not only explored the scaling of health and social care innovations but also any aspect of or any topic relating to such scaling, including (but not limited to) concepts, models, analytical models, tools, cost or impact assessments, and user engagement data. We considered “scaling” as inclusive of all variants of spread, scale or scaling up, scaling out or scaling deep, horizontal, and vertical. By adopting a broad view, we aimed to cast a wide net and thereby increase the learning potential of the study. ⁴³

Comparator

There were no restrictions.

Outcomes

We considered all outcomes reported in the included reviews, including health outcomes. We sought any outcomes such as (but not limited to) patients/caregivers' and/or health care providers' and/or policymakers’ perceptions and experiences of barriers, facilitators, acceptability and feasibility of scaling innovations, impact (e.g., adaptability, efficacy, effectiveness), coverage (e.g., proportion of the target population reached by the scaling), adoption, fidelity, penetration, maintenance of the innovation, health outcomes (e.g., impact on morbidity, mortality), patient‐reported outcomes (e.g., quality of life, satisfaction), and health care resources (e.g., cost effectiveness of the scaled innovation, cost of staff resources).

We excluded the review if it reported only on the impact of an effective health innovation but was not about scaling.

Types of Study

We considered all types of review (quantitative, qualitative, and mixed methods) that addressed, synthesized, or summarized preexisting evidence in the field of scaling. We define a review as a synthesis of evidence that includes a clear research question (i.e., mentions at least the population and the intervention) and describes the methods used to identify and select the primary research studies (i.e., mentions at least one data source and keywords). ⁴⁴ , ⁴⁵ Reviews could include studies with any research design. We included all reviews that had made explicit their methodological approach, whether they had been conducted according to their chosen design or not. We excluded primary research studies, conference abstracts, comments, opinions, letters, and editorials.

Context/Setting

We included reviews in any type of health or social care setting located in any geographical setting. We define “health and social care” as follows: Health care consists of services provided in institutional or community settings, any form of access to a health‐related service (such as dental, podiatry or optical services), and access to health care practitioners (such as nurses, physiotherapists, or general practitioners). ⁴⁶ Social care consists of interventions that support frail or vulnerable individuals by meeting needs or enabling them to meet needs that arise as a result of physical, mental, or emotional impairment. ⁴⁷

Data Sources and Search Strategy

A systematic literature search was performed to identify published studies in the following electronic bibliographic databases: MEDLINE via Ovid, Embase, PsychINFO via Ovid, CINAHL via EBSCOhost, Web of Science, The Cochrane Library, Sociological Abstracts via ProQuest, Academic Search Premier via EBSCO, and ProQuest Dissertations & Theses Global.

Our information specialist (NR) developed an Ovid‐MEDLINE and Web of Science strategy with input from the project team and a second information specialist. An iterative revision process was conducted by the members of the research team. We used the following main concepts: “scaling,” “reviews,” and “health and social care.” We improved a search strategy for scaling developed earlier by members of our team. ³³ , ³⁴ We adapted the Canadian Agency for Drugs and Technologies in Health (CADTH) search filter for Systematic Reviews/Meta‐Analysis/Health Technology Assessment for this project. ⁴⁸ A second information specialist reviewed the search strategy using the Peer Review of Electronic Search Strategies tool. ⁴⁹ The final version was approved by the team members and then translated into the other databases. Details on the search strategies are presented in Additional File 2. No language restriction was applied. We searched from inception onwards until August 6, 2020. Articles not available electronically were ordered via interlibrary loan.

Additionally, to identify gray literature, we searched Google and the websites of the following organizations: WHO (WHO‐ExpandNet), the UK's National Institute for Care and Excellence, NSW Government (Health Australia), the Canadian Foundation for Healthcare Improvement (CFHI), the International Development Research Centre, the CADTH's Grey Matters checklist, the Agency for Healthcare Research and Quality, the Global Community of Practice on Scaling Development Outcomes, the Institute for Healthcare Improvement, the National Implementation Research Network, the State Implementation and Scaling‐up of Evidence‐based Practices Center, the Health Foundation, the Australian Prevention Partnership Center, and Scaling Pathways. For each term, we reviewed the first 100 reports, except in ExpandNet and the CFHI, for which we searched all the reports in their websites (Additional File 3). The search took place on July 20, 2021.

Selection Process

The search results were imported and stored in an EndNote X9 library for reference management and duplicate removal. ⁵⁰ , ⁵¹ The resulting records were exported to the Internet‐based system Covidence for the selection process. ⁵² Pilot screening of the titles and abstracts of 2.5% randomly sampled reviews was completed to ensure common understanding of the selection criteria (1: Is it a review? 2: Does its aim concern scaling or inform scaling? 3: Is it about health or social care?). Paired independent reviewers screened titles and abstracts against the eligibility criteria. When abstracts did not appear to meet the criteria for exclusion, were ambiguous, or were missing, the studies were retained and reviewed in full. The reviewers’ full understanding of the selection criteria was validated again before beginning the full‐text selection. The full‐text eligibility assessment was done by paired reviewers independently (ABH, KVP, and RCC). In case of an “unclear response” regarding the eligibility of studies, authors resolved disagreements through discussion and, if necessary, consulted a third senior author (FL or AG). Any reasons for exclusion were recorded in Excel.

Data Extraction

We developed a data extraction form based on the JBI form for review of systematic reviews. A pilot data extraction was conducted on five of the selected reviews to ensure correct comprehension of the grid. Paired independent reviewers conducted the data extraction (ABH and RCC). Any disagreements between the reviewers were resolved through discussion; if consensus was not reached, they consulted a third senior reviewer (KVP). We did not extract data from the primary studies included in the reviews. We summarized the review findings but not the results of primary studies included in the reviews. If the article required translation into English to enable data collection, we used DeepL Translator software. ⁵³ Extracted information included the following:

• Review characteristics: journal name; first author; language; year of publication; type of review as reported by authors (e.g., systematic, scoping, rapid, realist, literature review); source of funding; country of the corresponding author; aim of the study; country or region of interest (e.g., South Africa, Canada, sub‐Saharan Africa, LMICs); PICO or PICo; inclusion and exclusion criteria; diseases, conditions, or types of care of interest (e.g., HIV, malaria, mother and child health, cancer, diabetes); scaling terms used in the search (e.g., scal*, spread, large‐scale, diffusion, scalability); number and type of databases sourced and searched (e.g., MEDLINE, CINAHL, Google, WHO website); date range of database searching; whether quality of the primary studies was appraised; appraisal instruments used to assess their quality (e.g., Grading of Recommendations Assessment, Development and Evaluation; Critical Appraisal Skills Programme; AMSTAR2, A Measurement Tool to Assess systematic Reviews 2; Mixed Methods Appraisal Tool); results summary in the abstract; review results (e.g., number of studies included, study design of primary studies included, number and characteristics of participants such as sex as reported, age); heterogeneity if applicable.

• Focus area of the reviews: The aim of the review and the results, if reported (yes/no), were organized based on previous reviews into the following focus areas: interventions, ³⁴ barriers or facilitators, ³⁹ frameworks or concepts, ⁵⁴ scalability, ⁵⁵ measurement, ⁵⁶ infrastructure ⁵⁷ , economic evaluation or costs, ⁵⁸ and other areas (see Additional File 4 for complete list and definitions).

• Scaling characteristics, in all reviews: How authors defined “scaling” (the source and the citation); mention of “sustain” or “sustainability” (yes/no); tool development (yes/no; if yes, tool name); mention frameworks related to scaling (yes/no), whether used in the methods or developed (yes/no; if yes, framework name).

For reviews focusing on the three most frequent areas, we extracted further data:

• Focus area: barriers and facilitators. Reviews that reported results on barriers and facilitators related to the following scaling domains: C1) health care infrastructure (e.g., providing medical equipment or changing procedures within a health system), C2) policy and regulation (e.g., revising policy to allow widespread community‐based case management of a disease), C3) financing (e.g., changing payment mechanisms, funding), C4) human resources (e.g., training and deployment of health care providers, changing roles of administrators), and C5) patient or public engagement (e.g., involving patients/public in recruitment or promotion). Scaling domains C1 to C5 were based on Ben Charif's five components of scaling strategies. ³⁴

• Focus area: interventions. Reviews that reported intervention results (yes/no) also related to scaling domains C1 to C5, i.e., health care infrastructure, policy and regulation, financing, human resources, and patient or public engagement, ³⁴ as well as relating to scaling acceptability, adoption, appropriateness, coverage, cost, feasibility, fidelity, and sustainability. ⁵⁹

• Focus area: scalability. Reviews that reported results on scalability also related to scaling domains C1 to C5, i.e., health care infrastructure, policy and regulation, financing, human resources, and patient or public engagement. ³⁴

Assessing Quality of Reviews

Two independent reviewers used AMSTAR2 ⁶⁰ to assess the methodological quality of all included reviews (ABH and RCC). Disagreements between the reviewers over particular studies were resolved by discussion, with involvement of a third author when necessary (KVP). The AMSTAR2 is a validated, widely used instrument for assessing the quality of systematic reviews of both randomized and nonrandomized trials of health interventions. It consists of a set of 16 questions about characteristics of systematic reviews, e.g., components of PICO, information on studies that were excluded, and consideration of risk of bias and how it was handled in the interpretation of results. The evaluator answers “yes” or “no” for the presence of each characteristic. A “partial yes” is allowed in some instances (Additional File 5). We adapted criteria 1 (about the components of PICO) and 8 (about describing the included studies) by adding the options “yes*” and “partial yes*” when all the elements were present except for the comparator. AMSTAR2 was not designed to generate an overall score, but an overall ranking can be performed with the seven most critical criteria (in bold in Additional File 5). We did not include scores for criteria 11 and 15 because they referred to meta‐analyses, which were performed in only two reviews. We used AMSTAR2 with all review types because we consider that methodological items 1 to 10 should be respected even in nonsystematic studies (e.g., realist reviews, narrative reviews). However, no overall score was generated for nonsystematic reviews. For systematic reviews, a score of “high” was attributed when none or only one noncritical criterion was unmet, “moderate” when more than one noncritical criteria was unmet, “low” when there was one critical flaw with or without unmet noncritical criteria, and “critically low” when there was more than one critical flaw. Reviews were not excluded based on their quality assessment.

Data Synthesis

We used the 2020 PRISMA flow diagram to describe the process of study selection. ⁴² We used tables to describe detailed characteristics of included studies (e.g., first author's name, year of publication, study aim, method details), analysis details (e.g., results summary, results details, whether authors reported heterogeneity). The data were charted using frequencies for year of publication, country of corresponding author, sources of funding, language, review type, and scaling results. In our tables we synthesized the following data as “yes,” “no,” or “not reported”: authors stated a PICO, the search used the term “scaling,” timeframe for search was indicated, appraisal included study quality and whether they used quality or risk appraisal instruments, and all scaling characteristics. For the funding sources, we subclassified governmental funding into two groups. “Governmental research councils” included all organizations specialized in funding research, e.g., national research councils such as the Canadian Institutes of Health Research and the US National Institutes of Health, whereas “Governmental/intergovernmental organizations” included government bodies or ministries whose role included funding research among other activities, and government or international development agencies (e.g., USAID, WHO). We classified the targeted countries within the reviews as well as the countries of corresponding authors (using authors’ stated affiliation) as HICs or LMICs based on the Organisation for Economic Co‐operation and Development income levels. ⁶¹ For the diseases or conditions, we classified under “HIV/AIDS” all reviews related to HIV or AIDS, whereas reviews related to malaria, tuberculosis, or influenza were classified as “other infectious diseases.” We classified all reviews related to maternal health, newborn or child health, pregnancy, breastfeeding, and family planning as “maternal‐child health.” Regarding the sources of documents searched, we recorded the number of databases searched and whether gray literature was searched or not.

We classified reviews into mutually exclusive focus areas based on their aims, i.e., reviews of barriers and facilitators, reviews of economic evaluation or scaling cost, reviews of frameworks, reviews of infrastructure, reviews of scaling interventions, reviews of measures, reviews of scalability, and emerging focus areas. The two emerging focus areas were bibliometric analysis and ethical, legal, and societal issues (ELSIs). If the aims related to multiple focus areas, the final classification was chosen based on the focus area most discussed in the review.

No meta‐analysis was performed, given that pooling the results of qualitative and quantitative reviews can introduce significant overlap and bias. Because our purpose was to synthesize evidence about scaling in health and social care in the literature, we present the whole body of knowledge and include the results of all the reviews, regardless of any overlap across primary studies. ⁶²

The deviation from the protocol ⁴³ was that we did not search clinical trial registries or the reference list of each included review, and we did not contact experts in the field to complete data collection as additional data sources. Given the fact that we excluded most of the gray literature for not qualifying as “reviews” according to our eligibility criteria, we judged it would have been only a marginal gain to contact experts for additional data sources.

Results

Review Search and Selection

Our electronic search identified 21,575 potentially relevant records. Of these, 120 reports met the review criteria, including 114 unique reviews and 6 associated reports. The gray literature search identified 2,694 reports. Thirty‐two were not available for full‐text examination; 23 unique reviews (plus 3 associated reports) were included (Figure 1). A reference list of included reviews is shown in Additional File 6. A list of excluded reports with the reason for exclusion is shown in Additional File 7.

Figure 1.

PRISMA Flow Diagram

Methodological Quality Assessment

The AMSTAR2 overall score was “critically low” for most of the systematic reviews (72%, n = 42/58) (Table 1, Additional File 5). Two scored “high” (one focusing on interventions, the other on costs) but did not respect the noncritical criterion (10) of reporting on sources of funding of included studies. Fourteen studies received the score “low,” 13 for not providing a list of excluded studies or reasons for their exclusion (critical criterion 7) and 1 for not using a satisfactory technique for assessing the risk of bias (critical criterion 9). Considering all types of reviews (n = 137), the criteria most respected were explaining inclusion criteria, describing included studies in adequate detail, and reporting conflict of interest. The least respected criteria were explicitly stating that the review methods were established before conducting the review and reporting funding of included studies. In the systematic reviews, the least respected criteria were assessing the risk of bias and taking it into account when interpreting results.

Table 1.

Quality Assessment AMSTAR2, as per Five Critical Criteria

Systematic Reviews a	n = 58	%
High	2	3.4%
Moderate	0	0.0%
Low	14	24.1%
Critically low	42	72.4%

^a This group includes the following types of review: systematic review, systematic review + meta‐analysis, hermeneutic systematic review + case study, and qualitative systematic review.

Review Characteristics

An overview of the 137 included reviews can be found in Table 2. Details of each review are presented in Additional Files 6 and 7. Except for one review, published in German, all reviews were published in English between 2005 and 2021, with a constant increase in publication frequency per year (Figure 2). The most frequent review type was systematic review (n = 58). The funding source was most commonly a governmental/intergovernmental organization (n = 31), followed by governmental research councils (n = 28). However, the majority of the reviews did not report a source of funding (n = 44). Corresponding authors were mostly from the United States (n = 43), the United Kingdom (n = 25), and Canada (n = 14). The majority of the reviews did not specify a country or a region of interest (n = 65) in their search methods; when one was specified (n = 72), they were mostly LMICs (n = 59). The corresponding authors of 71.2% of these 59 reviews focusing on LMICs were situated in HICs. The majority of authors did not state a clear PICO model (n = 121) but at least indicated inclusion criteria (n = 130) or exclusion criteria (n = 80). Most of the reviews were interested in scaling related to infectious diseases such as HIV/AIDS (n = 25); malaria, tuberculosis, or influenza (n = 11); or maternal–child health (n = 28). A significant number of reviews did not focus on a particular disease or condition (n = 47). The majority did not use scaling terms in their strategy search (n = 64). Most authors searched more than one database (n = 94) and the gray literature (n = 103). Most authors stated the timeframe of their search (n = 105). Few reviews assessed the quality of the studies they reviewed (n = 40). Most reviews reported data on study design (n = 90), but few reported on the number of study participants (n = 29), their age (n = 18), or their sex (n = 30).

Table 2.

Review Characteristics

	n = 137	%
Language of the publication
English	136	99.3%
German	1	0.7%
Year of publication
2005‐2009	11
2005	2	1.5%
2006	3	2.2%
2007	2	1.5%
2008	3	2.2%
2009	1	0.7%
2010‐2014	42
2010	9	6.6%
2011	4	2.9%
2012	8	5.8%
2013	10	7.3%
2014	11	8.0%
2015‐2019	65
2015	10	7.3%
2016	11	8.0%
2017	5	3.6%
2018	17	12.4%
2019	22	16.1%
2020‐2021 a	19
2020 a	14	10.2%
2021 a	5	3.6%
Review type b
Group of systematic reviews	58
Systematic review	54	39.4%
Systematic review + meta‐analysis	2	1.5%
Qualitative systematic review	1	0.7%
Hermeneutic systematic review + case study	1	0.7%
Group of nonsystematic reviews	79
Literature review	29	21.2%
Scoping review	9	6.6%
Rapid review	6	4.4%
Narrative review	5	3.6%
Comprehensive review	4	2.9%
Integrative review	2	1.5%
Qualitative review	2	1.5%
Realist review	2	1.5%
Structured review	2	1.5%
Critical interpretive synthesis	1	0.7%
Critical review	1	0.7%
Qualitative interpretive review	1	0.7%
Synthetic review	1	0.7%
Review + interviews or Delphi	14	10.2%
Sources of funding of the review c
Governmental/intergovernmental organizations	31	22.6%
Governmental research councils	28	20.4%
Charitable trusts	21	15.3%
Commercial organizations	1	0.7%
Mixed sources	12	8.8%
No source of funding reported	44	32.1%
Country of the corresponding author
HICs d	114	83.2%
United States	43	31.4%
United Kingdom	25	18.2%
Canada	14	10.2%
Australia	13	9.5%
The Netherlands	6	4.4%
Belgium	3	2.2%
Switzerland	3	2.2%
Denmark	1	0.7%
Germany	1	0.7%
Ireland	1	0.7%
Italy	1	0.7%
Scotland	1	0.7%
Sweden	1	0.7%
Switzerland and United States	1	0.7%
LMICs d	22	16.1%
South Africa	9	6.6%
India	4	2.9%
Bangladesh	1	0.7%
Brazil	1	0.7%
Congo	1	0.7%
Kenya	1	0.7%
Nigeria	1	0.7%
Pakistan	1	0.7%
Senegal	1	0.7%
Tanzania	1	0.7%
Zambia	1	0.7%
LMIC + HIC collaborations d	1	0.7%
China and Australia	1	0.7%
Income levels of targeted countries d
LMIC	59	43.1%
HIC	11	8.0%
LMIC + HIC	2	1.5%
Not specified	65	47.4%
The authors stated the elements of PICO e
Yes	16	11.7%
No	121	88.3%
Diseases, conditions, or care types of interest
Health care
Maternal–child health	28	20.4%
HIV/AIDS	25	18.2%
Other infectious diseases	11	8.0%
Mental health	4	2.9%
Cancer	2	1.5%
Chronic diseases	2	1.5%
Eye care	1	0.7%
Palliative care	1	0.7%
Social care and prevention
Physical activity	4	2.9%
Sexual education	4	2.9%
Violence	2	1.5%
Nutrition	6	4.4%
Reviews not focused on a particular disease
Health innovation regardless of disease	29	21.2%
Improvement the health system	18	13.1%
Search strategy used scaling‐related terms f
Yes	50	36.5%
No	64	46.7%
Not reported	23	16.8%
Timeframe for the search g
Yes	105	76.6%
Not reported	32	23.4%
Number of databases searched h
>1	94	68.6%
One database	28	20.4%
Not specified	5	3.6%
Not reported	10	7.3%
Gray literature
Yes	103	75.2%
Not reported	34	24.8%
Quality of the studies was appraised
Yes	40	29.2%
Not reported	97	70.8%

Abbreviations: HICs, high‐income countries; LMICs, low‐ and middle‐income countries; PICO, Participants, Intervention, Comparator, Outcome; WHO, World Health Organization.

^a Year incomplete, searched in databases August 6, 2020 and in gray literature July 20, 2021.

^b “Review type”: as described by the authors of their review; we did not verify whether the reviews met usual review characteristics of each review typology. “Literature review”: any review, review of published and gray literature, or evidence review.

^c “Governmental/intergovernmental organization”: government (e.g., Dutch government, health ministry), government or international development agencies, European organizations, USAID, WHO, World Bank, Global Fund. “Governmental research councils”: university grants, student grants, national research councils, Canadian Institutes of Health Research, National Institutes of Health, National Institute for Health and Care Research, National Health and Medical Research Council. “Charitable trust”: charitable foundations such as Grand Challenges Canada and Save the Children. “Mixed sources”: studies with more than one source of funding. “No source of funding”: studies did not report on funding sources or clearly indicated no grants or funding were received for the review.

^d Income country groups: based on the Organisation for Economic Co‐operation and Development Development Assistance Committee official list for development assistance⁵⁶ and on authors’ reporting of the countries they included or would target.

^e PICO models: We also included PICo: Population, phenomena of Interest, and Context.

^f Authors stated, either in the Methods or in an appendix, that they had used the following search terms, with or without the addition of “up,” “out,” or “down”: scale, scaling, spread, rollout, diffusion, large‐scale, system‐wide dissemination, and scalability. “Not reported”: authors did not report search strategy or none of these terms were used.

^g “Not reported”: author did not state the date range of their search.

^h The authors stated the search strategies used which databases or journal; “Not specified”: authors stated they would search peer‐reviewed journals or databases without mentioning their names; “Not reported”: used when any information about the databases or journal searched was found in the review.

Figure 2.

Distribution of Reviews per Year

Scaling Characteristics

The 137 reviews focused on seven areas: scaling interventions (n = 37), barriers and facilitators (n = 29), frameworks (n = 24), scalability (n = 24), cost (n = 14), measures (n = 5), and infrastructure (n = 2). Two additional focus areas added to our a priori list were bibliometric analysis (n = 1), and ELSI (n = 1) (Figure 3, Additional File 6). Only 36.5% (n = 50/137) of the authors defined scaling (Table 3). The WHO definition was the most frequently used (n = 26) (Additional File 8). Overall, most of the authors used the term “sustain” or “sustainability” (n = 104) (Table 3). In four reviews, a tool related to scaling was developed (a formula to calculate incremental coverage, ⁶³ the Becoming Breastfeeding Friendly Index, ⁶⁴ the Intervention Scalability Assessment Tool, ⁶⁵ and Scale‐up Readiness Benchmarks ⁶⁶ ) (Additional File 8). Of the 137 reviews, 73 cited at least one framework related to scaling, and 34 reviews developed their own framework or model (Table 4). A scaling framework was used in the methods of 30 reviews, with the ExpandNet framework the most frequent (n = 7).

Figure 3.

Distribution of Reviews by Focus Area

Abbreviation: ELSI, ethical, legal, and societal issues.

Gray boxes are additional focus areas added to our a priori list.

Table 3.

Results Related to Scaling

	n = 137	%
“Scaling” was defined
Yes	50	36.5%
No	87	63.5%
Mention of sustainability a
Yes	104	75.9%
No	33	24.1%
Developed a tool
Yes	4	2.9%
No	133	97.1%

^a The word “sustain” or “sustainability” was used.

Table 4.

Frameworks Related to Scaling

	Mentioned		Used in Methods		Developed
	n = 137	%	n = 137	%	n = 137	%
Yes	73	53.3%	30	21.9%	34	24.8%
In the focus area—Barriers/facilitators	14	10.2%	10	7.3%	2	1.5%
In the focus area—Costs	3	2.2%	0	0.7%	2	1.5%
In the focus area—Frameworks	24	17.5%	5	3.6%	20	14.6%
In the focus area—Infrastructure	2	1.5%	0	0%	1	0.7%
In the focus area—Interventions	14	10.2%	7	5.1%	6	4.4%
In the focus area—Measures	4	2.9%	1	0.7%	0	0%
In the focus area—Scalability	12	8.8%	7	5.1%	3	2.2%
No	64	46.7%	107	78.1%	103	75.2%

We explored some focus areas in more detail (Additional File 8). Under the focus area “frameworks” (n = 24), 5 reviews aimed to review existing scaling frameworks, 17 created a new framework, and 2 reviewed concepts of scaling (Figure 3). Under the focus area “barriers and facilitators” (n = 29), 27 reviews reported barriers or facilitators associated with health care infrastructure (C1) or human resources (C4). The least frequent barrier or facilitator reported was in the domain of policy and regulation (C2) (n = 23) (Table 5). Under the focus area “interventions” (n = 37), all reviews reported on the scaling domain of health care infrastructure (C1), and the least commonly reported was in the domain of policy and regulation (C2) (n = 22). Most reviews focusing on “interventions” reported on two outcomes: coverage (n = 16) and adoption (n = 10) (Table 6). Of the 24 reviews focusing on “scalability,” 23 reviews reported domains related to health care infrastructure, and the least frequently reported were domains related to policy and regulation (n = 14) (Table 5).

Table 5.

Scaling Domains Reported

	C1: Health Care Infrastructure a	C2: Policy and Regulation a	C3: Financing a	C4: Human Resources a	C5: Patient/Public Engagement a
In the focus area—Barriers and facilitators (n = 29)
Yes	27 (93.1%)	23 (79.3%)	26 (89.7%)	27 (93.1%)	24 (82.8%)
No	2 (6.9%)	6 (20.7%)	3 (10.3%)	2 (6.9%)	5 (17.2%)
In the focus area—Interventions (n = 37)
Yes	37 (100.0%)	22 (59.5%)	23 (62.2%)	36 (97.3%)	27 (73.0%)
No	0 (0.0%)	15 (40.5%)	14 (37.8%)	1 (2.7%)	10 (27.0%)
In the focus area—Scalability (n = 24)
Yes	23 (95.8%)	14 (58.3%)	15 (62.5%)	20 (83.3%)	17 (70.8%)
No	1 (4.2%)	10 (41.7%)	9 (37.5%)	4 (16.7%)	7 (29.2%)

^a As defined in Ben Charif and colleagues. ²⁹

Table 6.

Scaling Outcomes Reported

	Coverage		Cost		Feasibility		Acceptability		Fidelity		Adoption		Sustainability		Appropriateness
Yes	16	43.2%	7	18.9%	6	16.2%	7	18.9%	1	2.7%	10	27.0%	3	8.1%	2	5.4%
No	21	56.8%	30	81.1%	31	83.8%	30	81.1%	36	97.3%	27	73.0%	34	91.9%	35	94.6%

Discussion

We used an umbrella review to synthesize the evidence base for scaling in health and social care and found a total of 137 eligible reviews, among which 23 were from the gray literature. The vast majority of included systematic reviews scored critically low in terms of quality assessment, and many of the other review types did not respect the assessment criteria we considered important for nonsystematic reviews. We observed an increasing annual publication rate. The most frequent types of review were systematic reviews, and only two were meta‐analyses. Nearly one‐third of the reviews did not identify their source of funding. Among the 93 reviews that did report their funding, most were funded by governmental/intergovernmental sources. Although most first authors were from or based in HICs at the time of publication, most reviews were focused on scaling in LMICs. The most frequently covered clinical topic was infectious diseases, followed by maternal–child health, two important health issues in LMICs. Reviews focused on nine areas, most frequently on interventions, barriers and facilitators, frameworks, and scalability, whereas fewer reviews focused on measures, costs, or infrastructures. The most frequently used definition and framework were, respectively, the WHO definition of scale and its ExpandNet framework. Finally, within the three most frequent focus areas (barriers and facilitators, interventions, and scalability), the scaling domains of health care infrastructure and human resources were the most frequently reported on, whereas those related to policy and regulation were the least reported. Among reviews focusing on interventions, the outcomes most often reported were coverage, adoption, cost, and acceptability. These results lead us to make the following observations.

First, our study highlights the difficulty of using reviews (including this umbrella review) to draw conclusions about the impacts of scaling. The very low quality of the reviews and the heterogeneity of the scaled innovations may partly explain this. Even for the authors who claimed they were performing systematic reviews, the AMSTAR2 evaluation showed they did not respect the main requirements for this type of review. Combined with the small number of reviews that evaluated the quality of their included original studies, this failure to respect design modalities increased the risk of bias in our interpretation of their results. Defining scaling indicators or the use of standard implementation indicators would help identify common elements in scaling innovations and make meta‐analysis possible. This gap in methodology can be solved by being more systematic in reporting research results and by following reporting guidelines. ²⁰ Also, it indicates the need to develop and advance a more systematic and rigorous science of scaling, i.e., study it, publish on it, and publish better.

Second, increasing publication rates since 2005 can be explained by the relative novelty of the science and practice of scaling in the health and social care sector, especially in HICs. ¹ Even before the COVID‐19 pandemic, most health care systems across the world were struggling with increasing health‐related costs, ⁶⁷ , ⁶⁸ , ⁶⁹ higher consumer‐driven demands on health care systems, ⁷⁰ , ⁷¹ , ⁷² and a reduced health care workforce. ⁷³ , ⁷⁴ More and more health care systems are now looking for new ways to address these problems. One method, well known in LMICs, sometimes by another name (e.g., replication or expansion), is the scaling of effective evidence‐based innovations. In HICs, research funding has historically focused more on pilot projects and less on scaling, but this is changing as governments seek to reduce waste through economies of scale. ⁷⁵ Consequently, there has been an increase in government‐funded research into the science and practice of scaling in HICs ⁷⁶ , ⁷⁷ , ⁷⁸ , ⁷⁹ and hence more publications.

Third, as previous reviews have observed, ³³ , ³⁴ , ⁵⁵ , ⁸⁰ the vast majority of reviews that reported their region of interest (n = 72) focused on scaling initiatives in LMICs (n = 59). Although the cause of this finding was not clear from our results (we propose several hypotheses below), it suggests significant potential for South to North capacity strengthening, learning, and education. Why might the majority of scaling reviews focus on the Global South? LMICs need to deliver care at a national level under severe financial constraints and, as reflected in our findings, often use scaling to address their most urgent health challenges, especially infectious disease epidemics (e.g., HIV/AIDS, malaria, tuberculosis, or influenza) as well as health issues related to social inequalities (e.g., mother and child mortality). In addition, international agencies contribute expertise and funding to scaling in LMICs to tackle humanitarian crises and improve health indicators, partly with the goal of reducing global health threats. ⁷ Infectious disease programs in LMICs, such as providing access to antiretrovirals for HIV/AIDS or vaccinations for COVID‐19, may be faster to scale than chronic disease programs, and their effects may be more immediate and obvious than the effects of scaling programs for chronic diseases; as a result, they are published more frequently. Although scaling experience and expertise from LMICs are invaluable for HICs, the scaling challenges are different. However, LMICs are now also facing increasing challenges related to chronic illness. Scaling research that addresses a broader range of communicable and noncommunicable diseases is warranted.

Fourth, whereas almost half the reviews in our umbrella review focused on scaling in LMICs, two‐thirds of the first authors of these reviews were from HICs or based in HICs. Innovative LMIC scaling practices can become a source of learning for scaling in HIC contexts. ⁸¹ Authors located in HICs are clearly benefiting from the knowledge of LMIC experts to design, implement, and guide these scaling programs, as well as to gain funding and publish their findings. ⁸² , ⁸³ , ⁸⁴ These results highlight the need to fully acknowledge the important LMIC contributions to the field of scaling and to support the expansion of scaling research infrastructure in LMICs. Scaling strategies should be considered global knowledge, and an equitable relationship in South–North exchanges may help advance the evidence base as well as give due credit to the rich knowledge and experience of LMICs in scaling. In this regard, a group of researchers and innovators from across the Global South has recently issued a call to action to enhance visibility of Global South contributions to global communities working on scaling. ⁸⁵

Fifth, reviews covered all our a priori focus areas. Because scaling is a relatively new science, we expected to mostly find reviews about scaling interventions and reviews that analyzed scaling barriers, facilitators, and success factors. Costs and measures were the focus of fewer scaling reviews. These are significant knowledge gaps because both are elements valued by planners and policymakers when deciding whether to start scaling an innovation or not. Also, as previously observed, ⁵ , ³⁴ , ⁸⁶ there are few practical tools available for scaling preparation and execution. Gray, for example, reported that scaling programs used implementation indicators to measure progress or coverage, rather than scaling indicators. ⁸⁷ Another gap concerned equity. Only one review focused exclusively on ELSIs, showing that values‐related considerations in decision making about scaling, such as equity of care, community needs, and population participation, have been slow to make an appearance in the field. ⁷ , ⁸ , ³⁰ , ⁸⁸ Moral as well as technical justifications must be provided before, during, and after scaling is undertaken. ⁵ , ⁷ , ¹⁵ Updated conceptual frameworks for scaling should begin with moral justification as an essential pillar of scaling. ⁷ For example, in vaccine distribution, scaling considerations should include planning licensing schemes that ensure fairness and equity of access; otherwise, scaling can simply reinforce existing inequities. The evaluation of equity impacts must be dynamic and ongoing so that innovation characteristics can be adapted if necessary and unpredicted obstacles addressed in all scaling phases. ⁸⁹ In addition, health care systems are increasingly adopting the Quintuple Aim for Health Care Improvement, which maintains that all improvement and innovation efforts should focus on the individuals and communities who need them the most ⁹⁰ and moreover should involve them in the entire scaling process. ⁹¹

The novelty of the science also explains the lack of clarity about the definition of scaling and the frequent confusion between scaling and implementation or sustainability. Our finding that the WHO definition and its ExpandNet scaling framework were used to guide many reviews confirms that its influence is considerable in scaling innovations. However, a variety of definitions continue to be used in the broader field of scaling science. We suggest mixed interpretation of this finding. On the one hand, efforts to attain consensus on key terminology may help to clarify, unite, and advance the science of scaling. On the other hand, a diversity of scaling definitions may encourage diverse participation, cross‐fertilization, and qualitative richness for the field.

Finally, among reviews focusing on barriers and facilitators, scaling interventions, or scalability (the three most frequent focus areas), the factors or strategies most frequently reported on were in the domains of health and social care infrastructure and human resources. This body of evidence on establishing the appropriate infrastructures and best practices in capacity building could be indispensable to scaling researchers and practitioners seeking strategies for ensuring success. Examples of success factors included developing a scaling team within the organization to coordinate scaling with all the stakeholders, ² , ¹⁰ , ⁹² , ⁹³ putting in place beforehand all the infrastructure needed to realize the scaling, ³³ , ³⁹ monitoring the scaling process with indicators related to program objectives, ⁶⁴ , ⁹⁴ , ⁹⁵ and training health workers to improve their ability to use the innovation. ⁵⁷ , ⁹⁶ Since our previous review, ³³ , ³⁴ we observed an increase in the number of reviews that mentioned engaging the community and public and private partners as a scaling strategy or facilitator. Policy and regulation was the strategy the least reported, indicating a knowledge gap on this subject and highlighting the need for further reflection on the relation between scaling and macro systems. This gap possibly exists because it is more difficult for researchers to act on this strategy. It is important for scaling practitioners and researchers to know how to negotiate with macro policymakers, and they need evidence on how to face legal and/or governmental constraints in adapting their scaling interventions. ⁸⁹ However, although the issue of change management and governance is becoming more prominent, this gap could also exist because in fact the main bottlenecks in scaling are not at the policy or regulatory level but must be faced locally at the level of building the necessary infrastructure and addressing critical human resource issues. Indeed, current difficulties faced in HICs and LMICs have to do with a growing lack of human resources in relation to the increasing needs for health and social care. ⁹⁷

Our review has a few limitations. We may not have identified all eligible reviews because indexing in the science of scaling is an ongoing process. We did not seek the impact on our results of the overlap of primary studies included in the reviews because their research questions were different, but overlap may have resulted in overrepresentation in reporting of review characteristics (Table 2). Also, economic evaluations of some of the scaling programs may not have been included in reviews because such assessments are rarely published under the format of a knowledge synthesis and are rarely available other than as internal documents of the organization in question. Our search of the gray literature required that the reports be publicly available online. We assumed that economic evaluations would be no more detailed than in their organizations’ internal reports, which would be excluded for not describing their methods and therefore not meeting our study criteria. We recommend an environmental scan for this topic. However, our results are based on a large number of reviews and indicate important knowledge gaps.

Conclusions

The importance of scaling health and social care interventions has been reinforced by the COVID‐19 pandemic. A recent UN roadmap established implementation science as a key domain for COVID‐19 recovery, highlighting the urgency of examining how interventions that have been successfully implemented in one setting can be adapted to different contexts in ways that are effective and equitable. ²² This umbrella review, in which we review the evidence on the scaling of innovations in health and social care, is therefore timely. Our analysis shows where scaling is happening, which global regions are publishing on the topic, what scaling is focused on, and what scalers are using to guide their efforts.

The constant increase in publications confirms that scaling is a science in evolution. Nevertheless, the low quality of systematic reviews and the critical gaps found in most of their methodologies showed a need for better guidance and reporting in scaling research, or at least in review research.

Although LMICs have become experts in scaling to control infectious diseases and avert health problems related to economic and social crises, the authors of most publications were in HICs. Researchers must work not only toward true collaborations among patients, public, and stakeholders but also equitable partnerships between HICs and LMICs. As the science of scaling spreads across the world, it is essential to improve the power balance in research. All regions—both North and South—will suffer if this imbalance is not corrected.

We noted important evidence gaps regarding the type of innovations that are scaled. In LMICs, scaling efforts need to expand beyond controlling infectious diseases and averting humanitarian crises, whereas in HICs, scaling needs to move beyond the culture of perpetual pilot projects and address noncommunicable and chronic diseases. These gaps, combined with the advances in the science of scaling, suggest that it is time governments and funding agencies prioritize scaling not only health but also social innovations that address other complex conditions such those relating to aging and mental health. Programs in both LMICs and HICs also need to be scaled to reach more remote populations, such as Indigenous peoples, for prevention, screening, and treatment.

Above all, the aim of scaling health and social innovations must be to increase their impact including in reducing inequities and not just in reducing health care costs. ⁷ , ²⁸ , ³⁰ , ⁹⁸ The evidence of an increase in the engagement of patients and public in the scaling process and the emergence of ethical, legal, and societal considerations seem promising signals of this commitment.

Funding/Support: This research project is funded by a Canadian Institutes of Health Research (CIHR) Foundation Grant (#FDN‐159937). AG is funded by the CIHR Patient‐Oriented Research Awards. FL holds a Tier 1 Canada Research Chair in Shared Decision Making and Knowledge Translation. LW is supported by a National Health and Medical Research Council Investigator Grant (APP1197022) and Centre for Research Excellence (APP1153479). The financial providers were not involved in the project.

Acknowledgments: We thank Frederic Bergeron, MLS, for the peer review and evaluation of the electronic search strategies and for performing the search strategy. We acknowledge the precious work of the reviewers for the review selection and data extraction: Talagbe Gabin Akpo, Gloria Ayivi‐Vinz, Blanchard Conombo, Maya Fakhfakh, Lucas Gomes Souza, Felly Bakwa Kanyinga, Laura Langevin, Tania Lognon, and Yuxi Wang. We also thank Louisa Blair for her kind editorial help with the paper.

Supporting information