Skip to main content
. Author manuscript; available in PMC: 2021 Jan 1.
Published in final edited form as: Psychiatry Res. 2019 Aug 17;283:112521. doi: 10.1016/j.psychres.2019.112521

Table 1.

Quantitative Measurement Characteristics of Common Implementation Outcomes

Implementation outcome Level of analysis Other terms in the literature Salience by implementation stage Quantitative measurement method Example from the published literature
Acceptability Individual provider Individual consumer Satisfaction with various aspects of the innovation (e.g. content, complexity, comfort, delivery, and credibility) Early for adoption Ongoing for penetration Late for sustainability Survey Administrative data Refused/blank Auslander, W., McGinnis, H., Tlapek, S., Smith, P., Foster, A., Edmond, T., & Dunn, J. (2017). Adaptation and implementation of a trauma-focused cognitive behavioral intervention for girls in child welfare. The American journal of orthopsychiatry, 87(3), 206–215. doi :10.1037/ort0000233 https://www.ncbi.nlm.nih.gov/pubmed/27977284
Adoption Individual provider Organization or setting Uptake; utilization; initial implementation; intention to try Early to mid Administrative data Observation Survey Knudsen, H. K., & Roman, P. M. (2014). Dissemination, adoption, and implementation of acamprosate for treating alcohol use disorders. Journal of studies on alcohol and drugs, 75(3), 467–475. doi :10.15288/jsad.2014.75.467 https://www.ncbi.nlm.nih.gov/pubmed/24766759
Appropriateness Individual provider Individual consumer Organization or setting Perceived fit; relevance; compatibility; suitability; usefulness; practicability Early (prior to adoption) Survey Proctor, E., Ramsey, A. T., Brown, M. T., Malone, S., Hooley, C., & McKay, V. (2019). Training in Implementation Practice Leadership (TRIPLE): evaluation of a novel practice change strategy in behavioral health organizations. Implementation science: IS, 14(1), 66. doi:10.1186/s13012-019-0906-2 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6585005/
Feasibility Individual providers Organization or setting Actual fit or utility; suitability for everyday use; practicability Early (during adoption) Survey Administrative data Lyon, A. R., Bruns, E. J., Ludwig, K., Stoep, A. V., Pullmann, M. D., Dorsey, S.,… McCauley, E. (2015). The Brief Intervention for School Clinicians (BRISC): A mixed-methods evaluation of feasibility, acceptability, and contextual appropriateness. School mental health, 7(4), 273–286. doi:10.1007/s12310-015-9153-0 https://www.ncbi.nlm.nih.gov/pubmed/26688700
Fidelity Individual provider Program Delivered as intended; adherence; integrity; quality of program delivery Early to mid Observation Checklists Self-report Smith, J. D., Dishion, T. J., Shaw, D. S., & Wilson, M. N. (2013). Indirect effects of fidelity to the family check-up on changes in parenting and early childhood problem behaviors. Journal of consulting and clinical psychology, 81(6), 962–974. doi :10.1037/a0033950 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3852198/
Implementation cost Provider or providing Institution Payor Individual consumer Marginal cost; cost-effectiveness; cost-benefit Early for adoption and feasibility Mid for penetration Late for sustainability Administrative data Jordan N, Graham AK, Berkel C, Smith JD (2019). Budget impact analysis of preparing to implement the Family Check-Up 4 Health in primary care to reduce pediatric obesity. Prevention science, 20(5), 655–664. doi:10.1007/s11121-018-0970-x https://www.ncbi.nlm.nih.gov/pubmed/30613852
Penetration/Reach Organization or setting Level of institutionalization? Spread? Service access? Mid to late Case audit Checklists Emily M. Woltmann, M. S. W., Rob Whitley, P. D., Gregory J. McHugo, P. D., Mary Brunette, M. D., William C. Torrey, M. D., Laura Coots, M. S.,… Robert E. Drake, M. D., Ph.D.,. (2008). The Role of Staff Turnover in the Implementation of Evidence-Based Practices in Mental Health Care. Psychiatric Services, 59(7), 732737. doi :10.1176/ps.2008.59.7.732 https://www.ncbi.nlm.nih.gov/pubmed/18586989
Sustainability Administrators Organization or setting Maintenance; continuation; durability; incorporation; integration; sustained use; institutionalization; routinization; Late Case audit Checklists Survey Scudder, A. T., Taber-Thomas, S. M., Schaffner, K., Pemberton, J. R., Hunter, L., & Herschell, A. D. (2017). A mixed-methods study of system-level sustainability of evidence-based practices in 12 large-scale implementation initiatives. Health research policy and systems, 15(1), 102. doi:10.1186/s12961-017-0230-8 https://www.ncbi.nlm.nih.gov/pubmed/29216886
Quantity Organization or setting Proportion; quantity Early through late Administrative data Observation Brown, C. H., Chamberlain, P., Saldana, L., Padgett, C., Wang, W., & Cruden, G. (2014). Evaluation of two implementation strategies in 51 child county public service systems in two states: results of a cluster randomized head-to-head implementation trial. Implementation science, 9, 134. doi:10.1186/s13012-014-0134-8 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4201704/
Speed Organization or setting Duration (speed) Early through late Administrative data Observation Saldana, L., Chamberlain, P., Wang, W., & Hendricks Brown, C. (2012). Predicting program start-up using the stages of implementation measure. Administration and policy in mental health, 39(6), 419–425. doi:10.1007/s10488-011-0363-y https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3212640/

Note. This table is modeled after Table 1 in the Proctor et al. (2011) article.