Table 2.

Roadmap implementation and scale-up outcomes and measurement

Outcomes	Definition	Theoretical basis	Application to SCUBY	Assessment methods and tools
Roadmap implementation outcomes
Acceptability	The perception among implementation stakeholders that a given treatment, service, practice or innovation is agreeable, palatable or satisfactory.47	Cf. social validity67	Acceptable: (resource and implementation) stakeholders have mostly consensus or at least majority on way to go.	Surveys; key informants’ interviews
Feasibility	The extent to which a new treatment, innovation, strategy or programme can be successfully used or carried out within a given agency or setting.68	Cf. compatibility69	Feasibility signifies it is possible to reach the set goals specified within the roadmap.	Surveys; key informants’ interviews
Relevance	The perceived fit, appropriateness, or compatibility of the innovation or evidence-based practice (roadmap) for a given practice setting, provider or consumer; and/or perceived fit of the innovation to address a particular issue or problem.47	Cf. appropriateness, perceived fit47	Fit and relevance of the proposed framework, strategies, and actions to government policy agenda and stakeholder perception/interest.	Surveys; key informants’ interviews
Adoption	The intention, initial decision, or action to try or employ an innovation or evidence-based practice.47 Can be expressed as the absolute number, proportion, and representativeness of settings (contexts) and intervention agents (implementers) that are willing to initiate a programme (policy or intervention).	RE-AIM49; NASSS framework, Cf. non-adoption/abandonment15 50	Uptake of the proposed roadmap (element).	Policy dialogue reporting form; surveys; key informants’ interviews
Adaptation	The extent to which a policy or intervention is changed, the opposite of delivered as intended by its developers and in line with the programme model. It refers to the customisation and ongoing adaptation of the care package or programme model15; in this study, the adaptation of (preliminary and non-final versions of) the roadmap. Also linked to the concept of plasticity—‘the extent to which interventions and their components are malleable and can be moulded to fit their contexts’.48	MRC implementation fidelity28; plasticity48		Policy dialogue reporting form; surveys; key informants’ interviews; document reviews
Elasticity of the context	Elasticity can be defined as ‘the extent to which contexts can be stretched or compressed in ways that make space for intervention components and allow them to fit’.70	Elasticity48	Changes in the context that allow an acceleration or slow-down of roadmap strategies. Example 1: COVID-19 (slow-down because of other priorities, accelerator because of increased digitalisation efforts). Example 2: government change.	Follow-up stakeholder interviews (question on B&F); policy mapping on timeline (keep eye on policy developments and implications)
Scale-up outcomes
Coverage (horizontal scale)	The extent to which the target group is reached, in absolute and relative count.71	RE-AIM49 Cf. reach	Target population reached; number of people actually covered by the intervention. Example: people who have access to GPs/practices with improved ICP/ACIC score in Belgium. Number of HCs (population covered by HCs) implementing newly modified PEN package in Cambodia. Target group members reached with m-health and peer support intervention in Slovenia.	EHRs; population survey; health report/data; health facility assessment
Integration	Integration into health system and services (based on Meessen et al (2017), inspired by the universal coverage framework.25 72	RE-AIM49 Cf. maintenance Cf. penetration, institutionalisation, sustainability47	The extent to which complex systems (structure and processes) allow (maintain and institutionalise) ICP implementation. Example 1: through laws, regulation, financing. The level of institutionalisation of the recommendations in the roadmap. Example 2: in Belgium, health financing reforms and legal reform facilitating nurses in primary care. Example 3: in Cambodia, functioning NCD clinics and community-based peer support are linked to HC-PEN. Example 4: in Slovenia, integration of telemedicine and peer support for chronic patients’ management to primary care.	Key informants’ interviews; document reviews; Health facility assessment/ICP grid, EHRs
Expansion	Expanding the intervention programme (the ICP package to cover other elements). Similar to diversification as a type of scaling up in ExpandNet, also called functional scaling up or grafting, consists of testing and adding a new innovation to one that is in the process of being scaled up, hence, exploring the possibility of pilot testing an added component to the innovation.26	Cf. diversification26	Additional components in ICP; addition of comorbidities to package. Example: in Belgium, addition of nurses to GP practice; in Cambodia, newly modified PEN package; in Slovenia, addition of m-health and peer support to ICP.	Pre/post-ICP implementation evaluation via ICP grid appraisal of practices; key informants’ interviews

Tools can be found in the online supplemental appendices.

ACIC, assessment of chronic illness care; B&F, barriers and facilitators; EHRs, electronic health records; GP, general practitioner; HC, health centre; ICP, integrated care package; MRC, Medical Research Council; NASSS, non-adoption, abandonment, and challenges to the scale-up, spread, and sustainability; NCD, non-communicable disease; PEN, package of essential non-communicable disease interventions; RE-AIM, reach, effectiveness, adoption, implementation, maintenance; SCUBY, SCale-Up diaBetes and hYpertension care.