Table 1.

Iterative application and operationalization of RE-AIM for Sustainability, with a focus on health equity and dynamic context over time.

Reach
Indicators: Number, proportion, representativeness of individuals who participate in EBI. Key Questions: Who was the intended audience and who actually participated? Why or why not? How can we better reach them and engage with them? Health Equity Considerations: Are all populations equitably reached by the EBI? Who is not reached by the EBI (in terms of a range of social dimensions and social determinants of health) and why? How can we better reach those who are not receiving the EBI and ensure we are reaching those who experience inequities related to social dimensions and social/structural determinants of health? Sustainability Considerations: Who is/isn't reached by the EBI at various time points over time? (e.g., iterative measurement of Reach). Why or why not?
Effectiveness
Indicators: The impact of an intervention on important health behaviors or outcomes, including quality of life (QOL) and unintended negative consequences; consider heterogeneity of effects. Key Questions: Is the EBI effective? For whom? Are there any negative and/or unintended effects? Health Equity Considerations: Are the health impacts experienced equitable across all groups on the basis of various social dimensions and social/structural determinants of health- why or why not? Do certain groups experience higher levels of negative effects or burdens? Sustainability Considerations: Does the EBI continue to be effective at various time points over time? Among whom?
Adoption
Indicators: The number, proportion, and representativeness of: (a) settings; and (b) staff/interventionists who deliver the program, including reasons for adoption or non-adoption across settings and interventionists. Key Questions: Where was the EBI applied and by who? Which sites/staff were invited and which excluded? Which participated and not? Why? How can the setting/context/staff be better supported to deliver the EBI? Health Equity Considerations: Did all settings equitably adopt the EBI? Which settings and staff adopted and applied the EBI? Which did not and why? Were low-resource settings able to adopt the EBI to the same extent as higher-resource settings? What adaptations might be needed to facilitate adoption? Sustainability Considerations: Which settings/staff continue to deliver the EBI over time? Which do not and why?
Implementation
Indicators: At multiple setting and staff levels, continued and consistent delivery of the EBI (and implementation strategies) as intended (fidelity), as well as adaptions made and costs of implementation. Key Questions: Was the EBI and/or implementation strategies delivered consistently- why or why not? How was it be adapted and why? How much did it cost? How can we ensure the key functions of the EBI are delivered? Informed by existing implementation frameworks (e.g., PRISM, CFIR), what multi-level contextual determinants matter for implementation? Health Equity Considerations: Were the EBI and implementation strategies equitably delivered across settings/staff? Which settings/staff successfully delivered the EBI and implementation strategies and which did not and why? Do all settings/staff have the capacity and resources to deliver the EBI on an ongoing basis? What adaptations might be needed to promote equity and address social determinants of health? Sustainability Considerations: How do we ensure that the EBI continues to be delivered consistently over time, especially in the context of reduced funding? Are certain implementation strategies more likely to sustain EBIs and have sustained impact than others?
Maintenance/Sustainability
Indicators: Extent to which (a) health impact/benefits, outcomes, behaviors continue for patients/consumers at the individual level, including patterns in health inequities over time; (b) program activities or core components/functions of the original EBI (and strategies) continue to be delivered at setting/staff level with fidelity (e.g., continuation of active ingredients and essential functions/related activities), as well as the “evolvability” of the EBI and implementation strategies needed to support EBI delivery over time, including adaptations (planned and organic) and why they occur; (c) community and organizational capacity and infrastructure to deliver the EBI are maintained, including partnerships, networks, and coalitions; and when applicable (d) institutionalization, or extent to which EBI becomes part of routine organizational practices/policies (when considered dynamically over time) (all above measured initially 6 months after initial implementation and at least 1 year post EBI implementation and on ongoing basis, e.g., quarterly to annually). For the above, includes proportion and representativeness of settings that continue EBI and reasons why/not. Key Questions: What sustainability strategies can be used to sustain the program long-term beyond 1 year after implementation and longer? What are the costs and return on value of sustainability of an EBI? How can we support and incorporate the EBI so it is delivered past initial implementation or after the funding is over? Informed by existing sustainability frameworks (e.g., PSAT, ISF), what multi-level contextual determinants matter for sustainability? Health Equity Considerations: Is the EBI being equitably sustained? What settings and populations continue to be reached long-term by the EBI and continue to receive benefits over time- why or why not? Do adaptations to EBIs reduce or exacerbate health inequities over time? Do all settings have continued capacity and partnerships to maintain delivery of EBIs? Are the determinants of sustainability the same across low-resource and high-resource settings? How do social determinants of health shape inequitable implementation and sustainability of EBIs over time? Sustainability Considerations: As the program continues and the context and evidence changes, what adaptations (to the program, strategies, and setting) are needed to continue delivering the EBI long-term? Are there opportunities to build capacity at sites with low maintenance to promote longer-term sustainability? What would it take for sites to sustain the EBI over the long term? What are key multi-level barriers to continued program sustainability over time among a range of stakeholders? What are factors or strategies that might support continuation of the program? Over time as evidence changes, is de-implementation of some program elements a more appropriate outcome than continued delivery of the program? Are there sustainability strategies that are effective at maintaining impact and delivery over time?