Table 2.

Evaluation overview

TOOL	STAGE OF INTERVENTION	DATA SOURCE & COLLECTION	WHO PARTICIPATES	EXPECTED No. OF PARTICIPANTS	TIME COMMITMENT	INFORMATION NEEDED & JUSTIFICATION
Baseline Survey* (detail below table)	PRE & DURING	Online anonymous survey implemented via the ComPrac by the Social Scientist	Current or emerging executives or managers from all sites who enrol in the CompPrac and make decisions to improve the value of the programs, services, initiatives or technology for which they are responsible	80–120 across all sites	15 min	Baseline measure of individual values and confidence and organisational support and barriers around economic evaluation; as well as expectations about the embedded Economist online learning support. This survey will inform the learning support and the remaining embedded components of the program
Non-participant observations	PRE & DURING	Observation notes collected by Social Scientist	Economists and staff who planned or participated in the embedded Economist component of the intervention	Maximum 50 per site	PRE: 5 h maximum: 3 × 1 h scoping meetings 2 × 1 h co-production meeting DURING: a maximum of two hours per interaction for no more than four (4) meetings (minimum commitment 2 h maximum commitment 8 h)	Naturally occurring in situ group data about the process of the intervention (what was done by whom?), the relational aspects (how did the participants interact with one another?) and broader organisational context (i.e. what actions occurred and what decisions were taken in this environment) Provides a feel for organisational environments and the roles of actors within them that cannot be achieved through an interview situation.
Documentary analysis	PRE, DURING & POST	PRE: Meeting agendas and minutes Emails and telephone notes Draft operational plan Site decision making frameworks, policies, guides, tools, documentation developed by economist and sites including evidence of application DURING: Meeting notes, agendas, minutes, and associated documentation developed by the economist and sites including economic evaluations drafted, evidence of application and practice change from meeting agendas, minutes and site generated documentation will be collected POST: During the semi-structured interviews, participants will be asked to provide any documentation that demonstrate a change in processes and work practices as a result of the embedded Economist Project. Documentation may include meeting notes, agendas, minutes, policies, tools and guidelines	Site Leads and Economists and staff who planned or participated in the embedded Economist intervention will be asked to provide documents	Maximum 10 per site	2 h maximum (sourcing and providing documents)	Documentary evidence of how each stage of the intervention was formally enacted by the site and does this accord with what was observed and reported? Existing decision-making frameworks will also be analysed to ascertain the use of economic evaluation. Documentary analysis provides additional insights about the practices and around decision making and any practice outcomes from the intervention, thereby supporting any reported changes from interview data with tangible evidence
Semi-structured interviews (face-to-face or telephone)	PRE, DURING & POST	Interview transcripts and recordings	Economists and staff who planned or participated in the program	PRE: 5 staff 2 economists DURING: 25 staff 2 economists POST: 5 staff 2 economists	maximum 1 h per interview (N.B staff participants may be interviewed once during each phase of the intervention & economists may be interviewed up to 6 times during each phase)	Face to face anonymous inquiry providing in-depth descriptions and interpretations that are constructed and separated from the time (and place) in which the events take place about the process, context and relational aspects of each stage of the embedded component of intervention, including planning, how the intervention was enacted and the outcomes and impact in the post phase; as well as the learning support components at the time of the semi-structured interview. Participants will have the opportunity to review their transcript prior to data analysis.
Field diary	PRE, DURING & POST	Field diary from economists & staff	Economists and staff who planned or participated in the program	PRE: Lead economist DURING: Lead and embedded Economist 25 staff	Staff: Minimum commitment 15 min per week ×  12 weeks maximum commitment 1 h per week ×  12 weeks) Economist: Minimum commitment 15 min per week ×  15 weeks)	Reflections on how participants perceived and experienced each phase of the intervention. What did they do (process)? Who did they interact with (relational)? What broader organisational factors impacted each phase?
Education survey ** (DETAIL BELOW TABLE)	POST Completion of the university course	Online anonymous survey		Maximum of 60 across all sites		How are participants applying the knowledge gained from the university course in practice? Do they feel supported by their leadership to use evidence from economic evaluations? How are the learning supports contributing to practice change? i.e. have they increased capacity?