Implementation Preparation Costs of Virtual Reality Job Interview Training in Prisons: A Budget Impact Analysis

Abstract

Virtual Reality Job Interview Training (VR-JIT) has increased employment rates for returning citizens when added to a successful prison-based employment readiness program. However, implementation preparation cost—expenses prior to offering VR-JIT to intended recipients—is unknown. We estimated the cost of implementation preparation activities (e.g., organizing workflow) for two prisons to deliver VR-JIT. We conducted a budget impact analysis and enumerated the labor costs incurred during this important stage of implementation. Labor costs were approximately $8,847 per prison. Our sensitivity analysis estimated the labor costs to replicate this effort in a new prison to range from $2,877 to $4,306 per prison. Thus, VR-JIT may be an affordable tool for prison-based employment readiness programs to improve gainful employment.

Introduction

Each year over 500,000 incarcerated citizens return to their communities from state or federal correctional facilities (United States Department of Justice, 2021). However, returning citizens often struggle to reestablish their roles in society, with an estimated 79% being rearrested within 6 years (Alper & Durose, 2018). One major contributor to a successful reentry is obtaining gainful employment, consistent employment and sufficient earnings for returning citizens to be independent (Berg & Huebner, 2011). Research suggests employment reduces the risk of recidivism and supports returning citizens to secure housing, pay their bills, and establish a network in their communities (Gibson & Krohn, 2012; Petersilia, 2005; Ramakers, Nieuwbeerta, Van Wilsem, & Dirkzwager, 2017).

Although previous research suggests employment reduces recidivism, less than half of returning citizens successfully secure employment within 4 years post release (Seim & Harding, 2020; Visher et al., 2011). A systematic review suggests vocational programs may help improve employment outcomes, but such services are only available in about 50% of States (Newton et al., 2018; Stephan, 2008). In particular, the United States National Institute of Justice recommended that job interview training was a major gap in services (Wells, 2014), which aligns with the field’s lack of evidence-based job interview training (Flake, 2015) and that returning citizens struggle to discuss their employable strengths and the context of their prior conviction during job interviews (e.g., Pham et al., 2017; Ricciardelli & Mooney, 2018). Thus, there is a need to provide returning citizens support to enhance their job interview skills prior to their release to help expedite access to competitive employment upon community re-entry.

To address this gap in services, a recent pilot randomized controlled trial (RCT) found that 69% of returning citizens who engaged in a pre-release, trades-focused employment readiness program (called the Vocational Villages [services-as-usual]; Washington, 2018) were competitively employed by 6-month follow-up. Meanwhile, returning citizens at the Vocational Villages who were randomized to practice their interview skills with a novel intervention called Virtual Reality Job Interview Training (VR-JIT; a computerized job interview simulator with automated feedback; see methods for additional description) were competitively employed at 82% by 6-month follow-up (Smith et al., 2022). Moreover, the VR-JIT trainees within the Vocational Villages, as compared to the services-as-usual group, significantly improved their job interview skills and lowered their levels of job interview anxiety (Smith et al., 2022).

To validate whether VR-JIT is effective when delivered in prisons, a fully powered RCT evaluating the effectiveness and implementation process strategies of VR-JIT within prisons is underway (Smith et al., 2020). Notably, VR-JIT has demonstrated community-based effectiveness at enhancing employment outcomes for vocational services in special education and community mental health agencies (Smith, Smith et al., 2021; Smith, Smith et al., 2022; Smith, Sherwood et al., 2021; Smith, Sherwood et al., 2022). Based on these findings, the need to contextualize the labor requirements of adjunct employment services within prisons, and active interest in VR-JIT implementation by multiple state-level departments of corrections, we estimated the cost of the implementation preparation activities necessary to prepare a prison-based employment readiness program (at two prisons) to deliver VR-JIT. Implementation preparation is a well-defined phase of the implementation process that begins once a service delivery system decides to adopt a new intervention to the point when that intervention is offered to the intended recipient (Moulin et al., 2019). These costs are separate from the staff labor and costs required to actively implement and maintain their current programming, which are their own phases of implementation. Implementation preparation costs are germane given evidence that approximately half of all implementations that begin preparation activities fail to ever deliver the new intervention to a recipient (Saldana et al., 2012). Thus, the costs incurred are “sunk” as they fail to provide their intended benefit.

We also conducted a budget impact analysis and sensitivity analysis to estimate the labor and costs required to replicate the implementation preparation activities necessary for other prison employment readiness programs to deliver VR-JIT. The present study was conducted during our efforts to prepare two prisons to conduct the aforementioned RCT evaluating VR-JIT effectiveness (Smith et al., 2020). Overall, the results from this study could be useful to correctional administrators and state legislators when making informed decisions about educational tools to improve job attainment among returning citizens.

Method

Study Design

We determined the labor (and associated costs) and non-labor costs of preparing the two prison sites to implement VR-JIT within their pre-release employment readiness programs (i.e., the Vocational Villages within the Michigan Department of Corrections). These costs include planning meetings, training, and collaborations with the research and local school VIT-TAY/VR-JIT teams. The Vocational Villages are separate residential programs within the prisons where returning citizens live, work, and study together prior to their release. The curriculum focuses on preparing returning citizens for 13 vocational trades (e.g., CDL/forklift, carpentry, automotive) where returning citizens select one primary trade and earn legitimate trade credentials through coursework and hands-on training that are transferable to the workforce upon release. The Vocational Villages also require returning citizens to complete a 15-hour pre-employment workshop designed to enhance employability skills that includes job interview practice (Washington, 2018). The two prison programs annually served approximately 200 returning citizens each year.

We conducted a budget impact analysis (BIA) in which we surveyed the hours spent by research team members and prison staff on the activities required to prepare the prisons to implement VR-JIT (Sullivan et al., 2014). We used salaries and fringe benefits to calculate the cost of these activities and estimated costs to replicate the implementation preparation activities. The two Vocational Village programs were located at one minimum and one medium security prison in Michigan. The study was reviewed and approved by the University of Michigan Institutional Review Board (HUM000155161), and all prison staff provided informed consent.

Perspective

This BIA analysis was completed from the perspective of the prison budget holders and only includes the costs necessary for prisons to engage in the activities needed to prepare their staff and setting to implement VR-JIT within their ongoing employment readiness program.

Analytic frameworks

EPIS (Exploration, Preparation, Implementation, and Sustainment) is a comprehensive framework to guide the process to implement evidence-based practice in public health, social, health, and mental health services (Moullin et al., 2019). During the exploration phase, our research team partnered with the Michigan State Department of Corrections to consider programming needs (e.g., enhancing employment programming) and evaluating whether VR-JIT could help support those needs (Smith et al., 2020). During the preparation phase, we identified potential determinants of implementation (i.e., barriers and facilitators), and developed a detailed implementation plan to evaluate the effectiveness of VR-JIT. The current study is set during this implementation preparation period and our analysis focused on the resources and activities necessary for VR-JIT delivery as outlined in the EPIS framework. The implementation phase (i.e., delivery of VR-JIT and provision of implementation support) and sustainment phase (i.e., identification and facilitation of ongoing VR-JIT implementation supports) are beyond the scope of this study. For our computational framework, we used a cost-calculator approach that focuses on the collection of labor hours engaged in implementation preparation activities and the related salaries of personnel engaged in the activities (Sullivan et al., 2014).

Time Horizon

Implementation preparation took place from March 2019 through August 2019.

Participants

Participants in this study included members of the implementation support team, which consisted of the external scientific partner (n=6) and prison staff (n=9). The external scientific partner included the principal investigator, 3 research coordinators, 1 graduate student, and 1 information technology specialist. Prison staff included 2 administrative principals/leaders and 1 assistant principal (herein referred to as leaders), 2 teachers, and 3 information technology specialists from their respective prison.

Intervention

Virtual Reality Job Interview Training (VR-JIT; licensed by SIMmersion; www.simmersion.com) is an automated job interview simulator (with a Zoom-like interface) where participants repeatedly practice interviewing for up to 8 different jobs. Specifically, participants interview with a virtual hiring manager named Molly Porter whose mood can change based on an algorithm matrix of 9 personalities (e.g., friendly, inappropriate). Molly was created via video-recordings with an actress who recited thousands of lines of dialogue. Participants select their response to Molly’s question from a list of 10-15 scripted statements. The statements were generated through a collaboration between professional script writers and community stakeholders. The scripts were written at a 6^th grade reading level and range from highly effective responses to highly ineffective responses. Prior to training with Molly, participants reviewed 8 job interview skills (called learning goals; e.g., coming across as a hard worker or working well on a team). During interviews, the responses spoken to Molly through speech recognition software informed how well one performs the eight interview skills.

VR-JIT provides 4 levels of feedback via real-time non-verbal cues (from a coach in the corner of the screen), a transcript of specific answers to Molly’s questions, a performance assessment of the eight interview skills, and a numerical score from 0-100 based on the 8 interview skills. Trainees process this feedback and use it to improve their performance as they transition between easy, medium, and hard interviews towards completing 15 virtual interviews (i.e., the average number of interviews completed linked to improved employment outcomes during efficacy trials [Smith et al., 2015; Smith et al., 2017]). VR-JIT is typically accessed via the internet but was locally installed on 14 laptops (7 per site) for the prison-based RCT as internet-enabled devices were not allowed. Additional details about VR-JIT can be found here (Smith et al., 2020). The licenses for the ongoing trial were purchased all at once and cost $90 per VR-JIT license. This licensing framework is based on the 2018 pricing model that was budgeted for the grant application and may not reflect current pricing models. For more pricing information, please contact SIMmersion directly through their above website.

Data Source

Hours spent engaging in the various implementation preparation activities to prepare the prisons to implement VR-JIT were collected at the end of this phase. These data were collected using self-report surveys from prison and research staff using a timeline follow back procedure to complete hours for each month during the implementation preparation phase. All surveys were collected using electronic surveys via REDCap, a secure online data collection manager (Harris et al., 2009). Implementation preparation activity categories were created a priori by sharing categories used in a recently completed study (Smith et al., 2019). All participants reviewed the existing categories (from the prior study) and recommended adaptations to tailor the categories to be more representative of a correctional setting. Annual salaries and fringe benefits were provided by the Michigan Department of Corrections administrative team. Annual salaries for the research team were self-reported in 2019. Using the reported salary information, we then estimated fringe benefits to determine the cost of each labor hour and updated these to 2021 salary estimates so estimates were more comparable with current costs. Salaries included 2021 university and prison fringe benefits, which were estimated by the research team based on publicly available university data and fringe benefits directly from prison staff (Table 1). Non-labor costs for hardware and software were paid for and estimated by the research team.

Table 1.

Budget Impact Analysis Input Parameters

Variables	Input parameter	Median	Reference
Average salaries
External scientific partner (n=5)1	$ 106,5991	$ 58,880	Actual salaries2
MDOC Leaders (n=3)	$ 160,650	$ 160,650
MDOC Teachers (n=3)	$ 123,930	$ 123,930
MDOC IT (n=3)	$ 142,290	$ 125,460
Hardware
Computers (n=14)	$1,000
Headphones (n=14)	$15
Software
Software license (n=100)	$903

¹This input parameter includes annual salary information from 5 members from the scientific partner. One team member, a doctoral student, was paid $16 an hour and did not have an annual salary measure.

²Salaries include fringe benefits and are based on 2021 salary estimates.

³Software costs reflect the publicly available cost listed on www.simmersion.com. The website notes group discounts are available.

Measures

To calculate implementation preparation costs, we collected two measures: participant salary and participant time spent on specific implementation preparation activities (enumerated in Table 2). Participants reported the amount of time spent on each activity via a monthly self-report. Prior to data collection, the list of activities was adapted from prior research evaluating implementation preparation activities of VR-JIT at a community mental health agency (Smith et al., 2020). Prison staff reviewed the existing activities and identified which activities were relevant and added new activities specific to the prison setting. Then the research team finalized the activity list. One example of a listed activity was prison staff needed to complete an orientation on how to use VR-JIT, how to instruct participants to use VR-JIT, and a review of best practices. Another example is that prison IT staff needed to prepare computing devices so that they could safely be used within the prison.

Table 2.

Implementation Preparation Activity Labor Hours and Costs

	Total Number of Unique Staff	Total Hours	Proportion of Total Hours (%)	Proportion of Hours Accrued by the External Scientific Partner (%)	Total Labor Costs	Proportion of Costs Accrued by the External Scientific Partner (%)
Meetings and Correspondence
Delivery Planning
Meetings and correspondence among the implementation preparation support team members1	10	32.25	11.3	34.8	$ 2,011	34.3
Meetings and correspondence among the MDOC staff2	5	25	8.8	0	$ 1,701	0
Meetings and correspondence among the external scientific partner3	4	16.5	5.8	100	$ 665	100
Meetings and correspondence among MDOC staff and software team4	0	0	0	0	$ -	0
Physical infrastructure to support VR-JIT delivery
Meetings and correspondence among the implementation preparation support team members1	4	22	7.7	68.1	$ 1,275	67.3
Meetings and correspondence among the MDOC staff2	6	18.3	6.4	0	$ 1,344	0
Meetings and correspondence among the external scientific partner3	5	14.75	5.2	100	$ 904	100
Preparing to purchase
Meetings and correspondence among the implementation preparation support team1 and software team4	6	16.75	5.9	85.0	$ 1,264	45.0
Meetings and correspondence among the MDOC staff2	0	0	0	0	$ -	0
Meetings and correspondence among the external scientific partner3	2	8.5	2.9	100	$ 727	100
Meetings and correspondence among the external scientific partner3 and software team4	2	2.5	0.8	100	$ 224	100
VR-JIT Technology Setup: Time spent in meetings, corresponding, preparing computers, providing support
Delivery planning
Correspondence and on-site support among implementation preparation support team1	5	12.45	4.3	100	$ 1,098	100
Meetings among implementation preparation support team1 and software team4	4	8	2.8	100	$ 580	100
Meetings and correspondence among MDOC staff2 and MDOC IT staff	3	37.75	13.3	0	$2,523	0
Materials to deliver VR-JIT
Delivery planning
Review VR-JIT Training Materials among the MDOC staff2	5	26	9.1	0	$ 1,708	0
Developing, tailoring, reviewing, or printing materials (by external scientific partner3) to train MDOC staff2 to deliver VR-JIT	4	21.5	7.5	100	$ 770	100
Orientation/training
Delivery planning
Training and monitoring of MDOC staff2 to deliver VR-JIT by the implementation preparation support team members	4	7	2.4	100	$ 260	100
Complete VR-JIT orientation training among the MDOC staff2	4	4	1.4	0	$ 256	0
Prison orientation and prison security among the external scientific partner3	5	10.33	3.6	100	$ 380	100
Total	15	283.58	---	53.1	$17,694	47.0
Price per prison					$ 8,847

Note. MDOC = Michigan Department of Corrections, UM = University of Michigan, IT = Information Technology, VR-JIT = Virtual Reality Job Interview Training.

¹Implementation preparation support team includes the UM and the MDOC teams.

²MDOC staff include the MDOC leaders and teachers.

³External scientific partner is the UM team (principal investigator, graduate student, research coordinators, IT staff).

⁴Software team is SIMmersion and the MDOC IT staff.

Analysis

For our analysis, we used a cost calculator approach and utilized Microsoft Excel calculation commands to sum the total hours and compute the cost per activity (Sullivan et al., 2014). We calculated labor costs using each participant’s per-hour salary rate and the time spent on each implementation preparation activity. We estimated labor costs by multiplying the time spent on each activity by the per-hour salary of each participant (e.g., prison information technology (IT) specialist). Once calculated, the activities and their labor costs were sorted by activity type: meetings and correspondence, VR-JIT technology setup, materials to deliver VR-JIT, and orientation and training.

To calculate replication costs, the external scientific partner (i.e., the Michigan University research team) generated replication estimates based on their experience preparing the prison sites for VR-JIT implementation and previous implementation preparation expertise. Notably, time spent on activities focused on preparing the aforementioned RCT was excluded as these costs were unrelated to the VR-JIT implementation preparation activities.

Uncertainty analysis

To provide prison administrators (or budget holders) more confidence in the estimated costs, the external scientific partner also generated reasonable effort ranges to replicate each implementation preparation activity. To generate replication efforts, the principal investigator (MJS) and research coordinator BR met and discussed how much effort would be required to replicate the implementation preparation activities in new settings, given that efforts from the existing study may optimize the efficiency of future implementation efforts. These values were then used to conduct a sensitivity analysis and expected cost range for each activity (Sullivan et al., 2014).

Results

Table 1 presents the input parameters for labor and non-labor costs. Total non-labor costs summed to $23,210. The total labor cost of implementation preparation for VR-JIT in two prisons was $17,694, based upon 284 total labor hours (Table 2). The estimated labor cost per prison was $8,847. When labor and non-labor costs were summed together, implementation preparation costs were $20,452 per prison. Over half (56.7%) were from non-labor costs $11,605.

In Table 2, over half of labor hours (55.2%) were from meetings and correspondence for delivery planning, physical infrastructure to support VR-JIT delivery, and preparations to purchase computers. The remaining labor hours were distributed across VR-JIT technology setup (20.5%), orientation and training (18.3%), and materials used to deliver VR-JIT (16.7%). Table 2 displays all the implementation preparation activities grouped by category and includes the number of unique individuals in each activity, the total hours, the total labor costs, and the proportions of hours and costs accrued by the external scientific partner. The total hours recorded in Table 2 represents the total time spent on each activity and varied by staff member. For example, there were 26 total labor hours for the activity “Review VR-JIT Training Materials among the Michigan Department of Corrections (MDOC) staff”; however, this does not mean that the 5 staff members captured in this activity each spent 5.2 hours on this activity.

Table 3 displays the replication estimates and effort range for each implementation preparation activity. Overall, the estimated total labor replication cost for this study was $7,182 or $3,592 per prison (61% lower than the cost in the 2 prisons described above). We estimated the total cost of replicating the VR-JIT implementation preparation labor costs would range from $5,753 to $8,611, which is $2,877 to $4,306 per prison. Replication efforts are the amount of effort expected by future prison staff (outside of the context of a research study) to prepare a prison to implement VR-JIT (e.g., setting up physical infrastructure to use VR-JIT, updating computers to safely deliver VR-JIT). These cost estimates are based on replication efforts ranging from 5% to 100%, depending upon the task. For example, future correspondence regarding on-site technology setup support will likely consist of a lower effort in the future because the activities required for this task were unknown at the start of the study, and we have since developed a process that can be generalized (with minimal adaptation) to other prison settings. Additionally, the completion of VR-JIT orientation training among prison staff is expected to remain a high effort task because the training is standardized, and existing feedback did not suggest making adaptations to this process to shorten it. Notably, 7 implementation preparation activities received an estimate of 0% because each activity was not an expected cost (i.e., no staff labor hours required) to implement VR-JIT in future prison settings. These costs were only attributed to preparing for the very first VR-JIT implementation in prisons.

Table 3.

Implementation Preparation Assumption Estimates

Implementation Preparation Labor Costs	Estimate (%)	Effort Estimate Range (%)
Meetings and Correspondence
Delivery Planning
Meetings/correspondence among implementation support team1	25	20-30
Meetings/correspondence among MDOC staff2	40	30-50
Meetings/correspondence among external scientific partners3	15	10-20
Meetings/correspondence among MDOC staff and software development team4 (SIMmersion)	0	0
Physical infrastructure to support VR-JIT delivery
Meetings/correspondence among implementation support team	10	5-15
Meetings/correspondence among MDOC staff	70	55-85
Meetings/correspondence among external scientific partners	10	5-15
Preparing to purchase
Meetings/correspondence among implementation support team and software development team (SIMmersion)	35	25-45
Meetings/correspondence among MDOC staff	0	0
Meetings/correspondence among external scientific partners	0	0
Meetings/correspondence among external scientific partners and software development team (SIMmersion)	0	0
VR-JIT Technology Setup: Time spent in meetings, corresponding, preparing computers, providing support
Delivery planning
Correspondence/on-site support for implementation support team	10	5-15
Meetings among implementation support team and software development team (SIMmersion)	50	40-60
Meetings/correspondence among MDOC staff and MDOC IT	85	70-100
Materials to deliver VR-JIT
Delivery planning
Review VR-JIT Training Materials among MDOC staff	90	80-100
Developing, tailoring, reviewing, or printing materials to train MDOC staff to deliver VR-JIT among external scientific partner	15	10-20
Orientation/training
Delivery planning
Training/monitoring of MDOC staff to deliver VR-JIT among the implementation support team members	90	80-100
Complete VR-JIT orientation training among the MDOC staff	90	80-100
Prison orientation and prison security for external scientific partner	0	0
Non-Labor Costs
Computers	Required	-
Headphones	Required	-
Software license	Required	-

Note. MDOC = Michigan Department of Corrections, UM= University of Michigan, IT = Information Technology, VR-JIT = Virtual Reality Job Interview Training.

¹Implementation support team includes the UM and the MDOC teams.

²MDOC staff include the MDOC leaders and teachers.

³External scientific partner is the UM team (principal investigator, graduate student, research coordinators, IT staff).

⁴Software team is SIMmersion and the MDOC IT staff

Figure 1 displays the estimated cost ranges per implementation preparation activity. These activities are ranked from highest to lowest range. For most activities, the cost range was within ±$100 of the base value. Most other activities ranged between ±$100 and ±$200. The activity with the widest range (meetings and correspondence among MDOC staff and MDOC IT) was estimated to cost between ±$314. The activities with the largest estimated ranges involved MDOC staff, including meetings and correspondence among MDOC staff and MDOC IT for VR-JIT setup (±$314), meetings and correspondence among the MDOC staff about the physical infrastructure to support VR-JIT delivery (±$202), reviewing VR-JIT training materials among the MDOC staff (±$171), and meetings and correspondence among the MDOC staff about delivery planning (±$170).

Figure 1.

Discussion

Building on the initial effectiveness of prison-delivered VR-JIT at enhancing the ability to discuss a prior conviction and increased employment rate (omitted citation) and the field’s limited knowledge regarding implementation cost estimates of prison programs, this study estimated the associated labor and non-labor costs needed to engage in VR-JIT implementation preparation activities. Specifically, we found the cost to be $20,452 per prison, with over half of implementation preparation activities involving the nonlabor costs. We separately estimated labor costs to train staff, prepare workflow plans, and prepare physical space for VR-JIT delivery to be $8,847 per prison. After considering replication costs, we estimated the implementation preparation at new sites would cost approximately $3,592 in labor and may range from $2,877 to $4,306 per prison to support 100 returning citizens. In comparison, a meta-analysis found that educational programs for incarcerated adults cost between $1,400 and $1,744 per adult (Davis et al., 2013). Although not directly comparable, our findings suggest that VR-JIT may be a low-cost complement to other vocational services offered in prisons or low-cost service for prisons without other vocational programming. As state and federal policymakers and their correctional leaders make decisions involving services that may help to reduce recidivism, they should consider how the combination of existing vocational or work release programs and VR-JIT may support successful integration back into society.

This analysis provides an initial estimate but may vary beyond this range due to salary differences across states. Additionally, variation in prison staff involvement may also influence cost estimates. For example, if a deputy warden (assumed to have a higher salary) wants to get involved and attend trainings, this will increase costs. Our implementation preparation replication cost estimates assumed that future sites would request a high level of support to prepare the prison to deliver VR-JIT. For example, a high level of support includes training prison staff to: 1) instruct (with fidelity) returning citizens on how to use VR-JIT, 2) implement best practices for VR-JIT implementation, and 3) develop a plan for optimal VR-JIT delivery. Since this may not be the case for all sites, implementation preparation activities may cost much less than estimated in this study. Since prisons are low-resource settings, we offer a few suggestions to streamline or reduce costs to prepare prison settings to implement VR-JIT. First, prisons could adopt asynchronous training for their staff to learn how to deliver VR-JIT. Second, VR-JIT orientation training could be adapted into an automated process using self-guided videos. Third, multiple prisons within the same state system could simultaneously attend virtual trainings to reduce the labor of the VR-JIT orientation team.

Despite a high level of implementation support, our findings are comparable to previous implementation preparation activity estimates of VR-JIT in other settings. For example, labor costs for implementation preparation activities for VR-JIT within a large community mental health agency were $22,882 and the replication estimate was approximately $7,276 (68% lower than original cost; Smith et al., 2019), which is consistent with the 61% reduction in estimated replication costs for prisons in the current study. Initially, the estimated implementation preparation costs in the mental health agency appear to be twice as high as the costs in a correctional setting. However, when carefully comparing these two implementation preparation cost studies, we observed a few differences that may explain the estimated cost difference. First, several comparable activities were more expensive (with more hours) in the mental health agency setting as compared to the prison setting. For example, the activity “meetings and correspondence among the external scientific partner” took an estimated 100 hours in the mental health agency study, while it only took 17 hours in the prison setting. This disparity can be explained by the external scientific partner implementing lessons learned from this first implementation study, which facilitated a more efficient strategy to complete implementation preparation activities in the prison setting. Second, in the prison, 6 fewer staff members were involved in implementation preparation than in the community mental health agency, which reduced labor costs. Lastly, the prison settings were intended to support approximately 100 returning citizens (per site) within the context of the study, while the mental health agency setting for VR-JIT was intended to support approximately 200 individuals with serious mental illness.

Limitations and Future Directions

There are some limitations that need to be considered while interpreting the results. First, the cost estimates may not generalize to prisons in other states and at different security levels. Second, only two prisons participated in this analysis, which reduces the accuracy of our cost estimates when compared to the 1,000+ prisons in the United States (Sawyer & Wagner, 2020). Third, the research team purchased the computers for the prisons. As a result, our cost data did not capture any time the prison staff would spend discussing and planning the purchase of computing devices. Thus, we expect the implementation preparation cost estimate would be higher when prison staff are responsible for this task. Fourth, we used actual salaries to calculate labor costs. The salaries of prison-based educational and informational technology staff members may differ across states, which may affect estimating implementation preparation costs at other sites. Lastly, our sample only included prisons with employment readiness programs and, thus, may not generalize to prisons that lack these programs.

Of note, a few implementation preparation activities were not included in analyses because they were likely to vary widely or not be applicable to future VR-JIT implementation preparation activities. First, we did not include the cost of the external scientific partners traveling to prison sites to provide such support. Second, although a requirement for all non-prison staff visitors, time spent completing prison orientation activities (in order to safely enter the prison) was not included in the replication labor costs. These costs should be considered when interpreting the estimates from this study, as those costs would likely increase the costs of preparing to implement VR-JIT.

Although this study focused on estimating the costs of activities in the implementation preparation phase, future studies need to evaluate the costs of the VR-JIT implementation through all implementation phases (including sustainment), the cost-effectiveness of VR-JIT, and potentially compare the costs of different VR-JIT implementation strategies.

Conclusion

This study revealed that the implementation preparation labor costs of VR-JIT were estimated to cost $3,592 per prison and may range as low as $2,877 and as high as $4,306 for an infrastructure that can serve approximately 100 returning citizens per site per year. These estimated costs may increase or decrease depending on prison-specific labor costs (e.g., staff salaries, number of returning citizens served). Prison administrators can use these estimates to inform their decisions with regard to their potential implementation of VR-JIT.