Table 1.
Characteristics of measures of sustainability and sustainability determinants and overall psychometric and pragmatic quality scores
| Measure name | Alternate name | Designed explicitly for Sustainability | Other constructs covered | Theory/framework | Target population involved in development | Setting/s developed and validated in | Country validated in and languages available | EBI and setting designed to assessd | User perspective | Total PAPERS’ scoresa |
|---|---|---|---|---|---|---|---|---|---|---|
| Measures of sustainment | ||||||||||
| Provider Report of Sustainment Scale (PRESS) [32] | None | Yes | None | No | Yes | Substance use disorder treatment, child welfare, education, and specialty mental health |
Country: USA Language: English |
General EBI and setting | Executives and frontline staff | 35 |
| Stages of implementation completion (SIC) – Original [33–35] | None | No | Implementation process/stages | Unclear | Yes | Multiple |
Country: USA Language: English |
General EBI and setting | Researcher/purveyor | 14 |
| Measures of sustainability determinants | ||||||||||
| Assessment of Barriers to Implementation and Sustainability in Schools (ABISS) [36] | None | Yes | None | Framework proposed by the authors. | No | School |
Country: USA Language English |
Specific EBI: School-wide Positive Behavior Support (SWPBS) Specific setting: school |
Executive staff, frontline staff, and external personnel who have knowledge of a school's SWPBS systems. | 26 |
| Advanced Level Tier Interventions Treatment Utilization and Durability (ALTITUDE) [37] | None | Yes | None | No | Yes | School |
Country: USA Language: English |
Specific EBI: School-wide Positive Behavior Support (SWPBS) implementation at Tiers 1, 2, and 3 Specific setting: schools |
Executive staff, frontline staff, and external personnel who have knowledge of a school's SWPBS systems. | 21 |
| A measurement instrument for sustainability of work practices in in long-term care - long version [38] | None | Yes | None | Framework proposed by the authors. | Yes | Nursing homes, elderly homes, home care and care for disabled |
Country: Netherlands Language: Dutch English |
General EBI and setting | Executive and frontline staff | 26 |
| A measurement instrument for sustainability of work practices in in long-term care - short version [38] | None | Yes | None | Framework proposed by the authors. | Yes | Nursing homes, elderly homes, home care and care for disabled |
Country: Netherlands Language: Dutch English |
General EBI and setting | Executive and frontline staff | 22 |
| Change Process Capability Questionnaire (CPCQ) [39] | None | No | Change management | Framework proposed by the authors on organisational change | Yes | Medical groups treating patients with depression |
Country: USA Language: English |
Specific EBI: Depression care Specific: Clinical/healthcare setting |
Executive staff | 19 |
| Clinical Sustainability Assessment Tool (CSAT) [40] | None | Yes | None | Unclear | Yes | Multiple clinical settings |
Country: USA Language: English |
General EBI Specific setting: Clinical/healthcare setting |
Executive and frontline staff | 28 |
| Faith-Based Organization Health Integration Inventory (FBO-HII) [41] | None | Yes | None | No | Yes | African American Churches |
Country: USA Language: English |
Specific EBI: Health promotion activities Specific setting: Faith based organisations |
Executive staff | 17 |
| General Organizational Index (GOI) [42, 43] | None | No | Quality of implementation. | No | Yes | Community mental health centers |
Country: Norway USA Language: English Norwegian |
General EBI and setting | Researcher/purveyor | 24 |
| Levels of Institutionalization (Loln) [44, 45] | None | Yes | None | Level of Institutionalization Matrix framework [46] | Yes |
Organisations that operate health promotion programs - public schools, county health departments, and non-profit health agencies Hospitals and home health agencies |
Country: USA Language: English |
Specific EBI: Health promotion programs General setting |
Executive staff | 25 |
| National Health Service (NHS) Sustainability Model and Guide [47] | The National Health Service (NHS) Institute for Innovation and Improvement Sustainability Model | Yes | None | A broader sustainability model proposed by the authors | Yes | Health care settings |
Country: UK Language: English |
General EBI Specific setting: clinical settings |
Executive and frontline staff | 17 |
| The Normalisation Measure Development questionnaire (NoMAD) [48–53] | None | Yes | None | Normalization Process Theory [54] | Yes | Multiple clinical and community settings |
Country: UK USA Netherlands Canada Sweden Brazil Language: Albanian Danish Dutch English French German Italian Norwegian Portuguese Spanish Swedish |
General EBI and setting | Executive and frontline staff | 22b |
| New South Wales Sustainability Checklist [55] | None | Yes | None | None | Unclear | Health promotion programs |
Country: Australia Language: English |
Specific EBI: Health promotion workers General setting |
Executive and frontline staff | 16 |
| Office of Adolescent Health (OAH) Sustainability Assessment [56]c | None | Yes | None | The OPA Framework for Program Sustainability | Unclear | Adolescent health promotion and disease prevention |
Country: USA Language: English |
General EBI and setting | Executive and frontline staff | 16 |
| Office of Population Affairs (OPA) Sustainability Assessment Tool [57]c | None | Yes | None | The OPA Framework for Program Sustainability | Unclear | Adolescent health promotion and disease prevention |
Country: USA Language: English |
General EBI and setting | Executive and frontline staff | 18 |
| Prevention Program Assessment [58] | None | No | Organizational readiness for practice or program | Multiple theories | Yes | Public health departments, community organizations |
Country: USA Language: English |
Specific EBI: Chronic disease prevention programs General setting |
Executive and frontline staff | 27 |
| Program Sustainability Assessment Tool (PSAT) [10] | None | Yes | None | Public Health Program Capacity for Sustainability framework [13] | Yes | State and community settings |
Country: USA Language: English |
Specific EBI: public health practice and programs General setting |
Executive and frontline staff | 26 |
| Program Sustainability Assessment Tool (PSAT) – adapted for elementary setting [59] | None | Yes | None | Public Health Program Capacity for Sustainability framework [13] | Yes | School |
Country: Australia Language: English |
Specific EBI: public health practice and programs Specific setting: school |
Executive and frontline staff | 19 |
| Program Sustainability Index [60] | None | Yes | None | Framework of sustainability proposed by the original authors. | Yes | Community settings |
Country: USA Language: English |
Specific EBI: Community-based programs General setting |
Frontline staff | 23 |
| School-wide Universal Behavior Sustainability Index- School Teams (SUBSIST) [61–65] | None | Yes | None | Model of sustainability of school-based practices proposed by the original authors. | No | School |
Country: USA Language: English |
Specific EBI: School-wide Positive Behavior Support (SWPBS) implementation at Tiers 1, 2, and 3 Specific setting: schools |
Executive staff, frontline staff and external personnel who have knowledge of a school's SWPBS systems. | 29 |
| Sustainability Formative Self-Assessment Tool [66] | None | Yes | None | The Georgia Health Policy Center Sustainability Framework | Unclear | Community rural health settings |
Country: USA Language: English |
General EBI and setting | Executive and frontline staff | 15 |
| Sustainable Implementation Scale (SIS) [67] | None | Yes | None | A range of frameworks covered by current systematic reviews. | No | National sites |
Country: Sweden Language: English Swedish |
Specific EBI: Community mental health services General setting |
Researcher/purveyor | 23 |
| Sustained Implementation Support Scale [68] | None | Yes | None | No | Yes | Mix |
Country: Australia, Barbados, Belgium, Canada, Germany, Ireland, Netherlands, New Zealand, Romania, UK, United Arab Emirates, USA Language: English |
General EBI and setting | Executive and frontline staff | 23 |
| Sustaining Innovation Through Education (SITE): Extended Behavioral [69] | None | Yes | None | Author-derived framework | Yes | Schools |
Country Australia Language English |
General EBI Specific setting: schools |
Executive and frontline staff | 18 |
| Sustaining Innovation Through Education (SITE): short behavioral [69] | None | Yes | None | Author-derived framework | Yes | Schools |
Country Australia Language English |
General EBI Specific setting: schools |
Executive and frontline staff | 19 |
| Sustainment Leadership Scale [70] | None | Yes | None | Exploration, Preparation, Implementation, Sustainment (EPIS) framework [71] | Unclear | Community-based organizations |
Country: USA Language: English |
General EBI and setting | Frontline staff | 24 |
| Sustainment Measurement System Scale (SMSS) [72]* | None | Yes | None | Consolidated Framework for Implementation Research [73] | Yes | Substance Abuse and Mental Health Services |
Country: USA Language: English |
Specific EBI: prevention programs General setting |
Executive and frontline staff | 28 |
*Includes a sub-scale that assesses sustainability as an outcome and seven sub-scales assessing determinants of sustainability
a Higher total PAPERS score reflects a higher level of psychometric and pragmatic quality, with the highest possible score being 56
bTotal psychometric PAPERS rating for the Portuguese version of the NoMAD is 2, for the Swedish version (S-NoMAD) is 7, and for the Dutch version is 6, out of a possible 36. As the psychometric properties of translated measures may differ, the PAPERS ratings for these three translated versions of the NoMAD were scored separately
cThe Office of Adolescent Health (OAH) and Office of Population Affairs (OPA) amalgamated. However, both measures developed by these divisions are still publicly available and contain different content, thus were assessed separately
dThis is based on the intended focus of the measure and may differ to how the measure was originally developed and evaluated, which may reflect a more narrow focus than intended