Developing Computer-Based Learning on Care of Aged and Dying Incarcerated People

Abstract

Introduction

Corrections agencies are exploring ways to securely and cost-effectively increase access to high-quality, evidence-based educational programs for personnel. Technology-based instructional tools hold strong potential for continuing education. The Institute for Healthcare Improvement Framework for Going to Full Scale was employed to guide a systematic approach.

Purpose

The purpose of this article is to outline and describe the design and development of a media-rich interactive computer-based learning product, Enhancing Care for Aged and Dying in Prison (ECAD-P), which addresses geriatric and end-of-life (EOL) care issues in corrections.

Method

Through an iterative process the research team developed the computer-based educational program that included program and module-specific objectives in alignment with goals and priorities of the end users; detailed evidence-based content that was engaging and visually appealing; and assessments aimed at testing the user’s knowledge.

Results

ECAD-P contains six modules, created under the careful guidance of the research team and the two advisory boards. Content, including images and testimonials were selected purposefully and strategically. Module objectives were developed in alignment with the goals and priorities of each module and assessments tested user knowledge level pre/post module exposure. Completion of the training product advances the research and development necessary to further the goal of full-scale dissemination of the computer-based education.

Discussion/Conclusions

The goal of this program is to enhance care and improve quality of life for aged and dying inmates. Evidenced-based training products are critical in preparing not only forensic nurses who work in corrections, but also the broader group of correctional personnel in how to better meet the care needs of incarcerated persons.

Introduction

Corrections is a complex, hierarchical environment where all personnel are responsible for the care, custody, and control of incarcerated people. This includes providing care to those who are older and dying. For personnel to maintain, improve, and update knowledge and skills, continuing education is necessary (National Commission on Correctional Healthcare [NCCHC], 2018a; 2018b). In the free world, technologies have transformed the way we approach continuing education. Technologies such as computer-based learning (CBL) or e-learning programs are being used to provide distance learning and with less resources than in-person trainings (Rohwer, Anke, Motaze, & Young 2017).

Correctional institutions struggle to keep pace with technological advancements commonly available in the free world. Primary barriers include budgetary constraints, and the potential for security breaches (Jackson, Russo, & Hollywood, 2015). A breach in security could critically compromise custody and control, which are essential to safely managing the day-to-day operations. CBL could provide a sizable portion of the continuing education necessary, while minimizing the use of resources in corrections (e.g., decreasing the need for home-grown program development; providing a foundation for face-to-face discussions and practical “hands-on” applications as needed in facilities).

Despite these challenges, advancements are being made (Loeb, Penrod, Myers, Baney, Strickfaden, Kitt-Lewis & Wion, 2017). Examples of technologies that have been integrated into correctional settings are electronic health records, video visitation with outside family members, videoconferencing, and telehealth, as well as computer-based training for corrections personnel. The purpose of this article is to outline and describe the design and development of a media-rich interactive CBL product, Enhancing Care for Aged and Dying in Prison (ECAD-P), which addresses geriatric and end-of-life (EOL) care issues in corrections.

This project is timely because the percentage of older adults who are incarcerated is growing (Stevens et al., 2018) including many who will spend their final days behind bars (Wion & Loeb, 2016). Second, corrections agencies are exploring ways to securely and cost-effectively increase access to high-quality, evidence-based educational programs for personnel that can be delivered from a distance (Ellen Perrett, Erricker, & Lyons, 2014; Loeb, Penrod et al., 2017). According to the International Centre for Prison Studies (2004), training requirements vary considerably across nations. In the United States (US), the Department of Justice (2015) requires that employees must complete 40 hours of training per year. This number of hours is comparable to what is publically posted by the Michigan Department of Corrections (DOC) (2019), which specifies that 40 hours are required for staff who have contact with incarcerated people. In addition, 40 hours has been shared by some of our team’s advisory board members who are not affiliated with the Michigan DOC. Although there are mandated training foci (e.g., Prison Rape Elimination Act; Fire Safety), there are often remaining hours among the required 40, which are not designated for a specific training topic (i.e., approximately eight hours). This flexibility permits a particular correctional system (or even individual staff in some cases) to devote a portion of annual training hours to topics of particular institutional need and/or disciplinary focus. Expanding the professional development resources available for training officers in corrections and providing them with technology-based instructional tools holds strong potential for enhancing the educational experience of corrections staff. When developing training programs, it is important for researchers to consider the culture of corrections. Corrections personnel rely on rigid organizational regulations to make on-the-job decisions and may take a cautious approach to engaging in new activities out of concern about jeopardizing their employment or personal safety. Understanding the routinized complexity of corrections is essential to developing training materials for corrections personnel (Clear, Reisig, & Cole, 2018). In addition, Stevens and colleagues (2018) report in their systematic review of elder prisoner care interventions that justifying programs with both prison staff and prisoners is essential if legitimacy and acceptance are to be achieved.

The university team previously developed a paper-based End-of-Life (EOL) Toolkit for training corrections personnel (Loeb, Wion, Penrod, McGhan, Kitt-Lewis, 2018; Penrod, Loeb, Ladonne, & Martin, 2016). In a second study (Phase I), the paper-based educational materials were translated into an interactive CBL program (Loeb, Penrod et al., 2017; Kitt-Lewis, Loeb, Myers, Wion, Baney & Strickland, 2019). Throughout this process, the collaborative team capitalized on the complementary expertise of team members from a college of nursing at a major research university, a health communication research and technology company (referred to throughout as technology company), and an Expert Advisory Board (EAB). Geriatric content was developed, EOL content was extended, and the Toolkit was redesigned (Loeb, Penrod et al., 2017). When developing our modules we referred to sources including, but not limited to the End-of-Life Nursing Education Consortium (ELNEC) training (ELNEC, 2019), systematic reviews on end-of-life (Wion & Loeb, 2016) and geriatric care in prisons; the 2018 National Commission on Corrections Healthcare standards for prisons and jails (NCCHC 2018a; 2018b); papers reporting on model prison programs (e.g., Gold Coats program [Berry et al., 2016]; and the Angola Prison Hospice [Evans, Herzog, & Tillman, 2002]). The technology applied in the Phase I study was selected for its utility within the constraints of corrections environments. In-person usability testing of the prototype with prison multidisciplinary front-line staff (n=16) was conducted in two state prisons to evaluate the user interface, ease of use, and perceived barriers. Phase I established proof of concept and feasibility (Kitt-Lewis, Loeb, Myers et al., 2019).

Despite these accomplishments, further examination of scalability and scale-up was needed prior to full-scale dissemination of CBL modules. This is being explored in a Phase II project. Expanded testing of the CBL system will establish the effectiveness of the learning system and provide critical insights relevant to dissemination.

Framework

The Institute for Healthcare Improvement (IHI) Framework for Going to Full Scale was employed to guide a systematic approach to scaling up the ECAD-P training (Barker, Reid, & Schall, 2016; Figure 1) so it could be well prepared for mass dissemination. This Framework addresses the phases of going to full scale, the adoption mechanisms (i.e., contextual influences), and support systems (i.e., infrastructure) needed to achieve scale-up and sustainability (Barker et al., 2016).

Figure 1:

The Institute for Healthcare Improvement (IHI) Framework for Going to Full Scale

This is the framework that guides our training development.

Methods

A collaborative team was essential in building reliable processes that support sustainability, which include the Framework elements, adoption mechanism, and support system (Barker et al., 2016). In an iterative process, each group played different roles in product design and development. The collaborative team included researchers with expertise in geriatrics, EOL care, and corrections health; a multimedia team; and consultation from an EAB comprised of recognized experts in geriatrics, EOL care, and corrections; and a Community Advisory Board (CAB) comprised of relevant stakeholders (e.g., potential users; potential purchasers; and returning citizens [RC]). Returning citizens are defined as formerly incarcerated people (The Center for Returning Citizens, nd; United States Attorney’s Office, Northern Office of West Virginia, 2018). The seven CAB members were comprised of the following: a Director of Nursing for a State DOC; a Director of Quality Improvement Nurse from a large urban jail; a psychologist engaged in private corrections consultation; a Correctional Healthcare Administrator at a State Correctional Institution; a Director of Training from a State DOC; and two Returning Citizens (i.e., formerly incarcerated people). Precise ages were not collected; however, all were middle age or older adults. They worked in four different US states and collectively had experience in Federal and State prisons, and a large city jail. Two CAB members were African American (i.e., one male and one female) and 5 were Caucasian (four males and 1 female).

Throughout the development of the strategies, special consideration was given to contextual influences and the availability of supporting infrastructures in corrections. Contextual influences include such things as the degree to which a correctional facility balances its security focus with a treatment focus and the perceived relevance of a particular training topic to a correctional setting (e.g., are there sufficient numbers of incarcerated people who are aged and/or with advanced chronic health conditions to justify investing in staff training on care of aged and dying inmates and do decision makers believe that training to enhance the management and care of these special populations is important). For example, despite the prevalence of mental illness in correctional populations, staff typically do not receive adequate training on this topic (Kolodziejczak, & Sinclair, 2018). Supporting infrastructure includes such things as: financial resources (e.g., adequate budget to pay staff to complete training during work hours) (Loeb, Penrod, et al., 2017); human resources (e.g., sufficient staff to allow for rotating off post to engage in training); and the technological capacity to deliver computer-based training on site (Loeb, Penrod, et al., 2017). Taken together, perceived importance and need, along with capacity to meet the need are pressing concerns, which are amplified within the necessarily restrictive environment of corrections. With the goal of gaining greater contextual understanding and developing training that is tailored to the corrections environment, the research team fostered relationships with colleagues in corrections (EAB and CAB). The end-product will be developed to meet the market demands and adapted to the capacity of correctional settings.

Computer-Based Learning Program

The three previously developed prototype modules (Raising an Awareness, Delivering Supportive Care at the End-of-Life, and Dealing with Loss and Grief) and the usability feedback from Phase I (Loeb, Penrod, et al., 2017; Kitt-Lewis, Loeb, Myers et al., 2019) were reviewed by the EAB, CAB, and research team. The entire group collaboratively identified additional content topics; and the final content areas were reviewed through Qualtrics surveys, a teleconference and in-person meetings held over a six month period. Supportive care at the EOL tailored for healthcare professionals, geriatrics in prison, and innovative strategies to enhance care for incarcerated people who are aged and/or dying were the three new content areas that would be developed.. The Toolkit (Loeb, Wion, McGhan, Kitt-Lewis, & Hollenbeak, 2018), Phase I modules, and reputable sources (i.e., evidence-based practices, standards of care, research findings) were used to establish a detailed outline of potential content for the six targeted content areas. Refinement of the content of the three prototypical modules was conducted, an outline for each of the three new modules was developed, and credible resources were identified and obtained. Team members were assigned modules relevant to their area of expertise and then reviewed/revised existing content and developed new content. This iterative process of refinement resulted in consensus on the content for each module and is presented in a prior publication (Loeb, Wion, Penrod et al., 2018).

Objectives and Assessment

Objectives were tailored to the intended level of complexity and specificity of knowledge to be gained. For example, Bloom’s Revised Taxonomy objectives were generated at the level of “apply” or higher (Anderson et al., 2001; Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956). After the application of Bloom’s Revised Taxonomy, minor revisions were needed in the areas of level of complexity and specificity to be tested as well as comparison of objectives to content. General objectives to capture the goals of the CBL program were developed to align with the module objectives. However, upon further reflection and discussion, several concepts did not lend themselves to higher-level assessment and therefore lower-level objectives were retained (e.g., identify the three core values of ECAD-P, identify characteristics that indicate a need for Supportive Care; list team members who are involved in providing Supportive Care). The final objectives were reviewed by the EAB and three master’s prepared academic nursing faculty (all of whom had experience in geriatrics or EOL care).

Images

Images can be powerful teaching tools that can accelerate learning. A primary goal in the development of the interactive CBL modules was to incorporate images that were visually stimulating to promote active, engaged learning. Several methods were used to ensure this goal. The CAB (n=7) members were asked to review 26 images and provide their expert opinions on the realism and emotional impact t of each one through a face-to-face discussion. Based on the CAB feedback, an extensive internet search was conducted to locate additional images that captured the phenomena of focus in each module and were congruent with the context of corrections. A compilation of images was developed, and CAB members were charged with independently selecting three images in rank order that best portrayed the intended phenomenon or concept of focus. Photo selection was administered via two Qualtrics surveys. Two surveys were necessary due to the large number of photos. Photo survey 1 (modules 1–3) and Photo survey 2 (modules 4 and 5) contained 198 images and 124 images, respectively. The Qualtrics surveys were distributed via email to both the CAB and EAB members (n=11); however, not all 11 members completed each of the surveys. Our goal was to obtain at least 4 responses for each survey; however, we exceeded that goal with nine completing Photo Survey 1 and five completing Photo Survey 2. Each member determined “How well do the following images portray (or capture the notion of) this concept?” Once photos were selected, permissions to use the photos were secured.

During the image search process, the collaborative research team (i.e., CAB, EAB, and university research team) recognized that due to the challenges related to securing all the desired images (e.g., photographers not granting use permission, high cost to purchase photo) some CBL modules would lack the quality of visual images desired by the collaborative research team. This shortcoming was particularly evident for Module 3: Loss, Grief, and Bereavement and Module 5: Innovative Strategies in Action. In response, other options were considered. This challenge provided an opportunity which inspired the team to consider alternate strategies, such as creating our own images through a staged photo shoot; filming video testimonials of prison insiders/former insiders; or enlisting the services of a graphic artist. After examining the logistics of creating our own staged images (i.e., securing a correctional facility, gaining access and permission, identifying a photographer, creating scenarios, organizing a prop list, and identifying models), along with considering our EAB and CAB’s lack of endorsement for photos that appeared to have been staged, the collaborative research team decided this was not the best approach. Consultation with our CAB resulted in embarking on a two-pronged approach. First, we would engage RCs as consultants to provide video-testimonials regarding their experiences with loss and grief while incarcerated (see details below). Second, the team determined that a graphic novel stylization (i.e., serious content presented in a high-quality illustration) would be an innovative, yet financially feasible approach to meeting the visual needs of some modules (Figure 2). A graphic artist was commissioned to create storyboards depicting the concepts outlined in two of the six modules. In addition to images, video testimonials, and graphic novel depictions, a logo for the ECAD-P CBL product was designed and then approved by the CAB.

Figure 2:

Example of Graphic Novel

This figure provides an example of a graphic novel stylization presented in Module 5.

Logo

The logo from the original paper Toolkit (Loeb, Wion, Penrod et al., 2018) was evaluated by the collaborative research team for fit with the CBL product. The color schema, wordy design, and the visual image were assessed by the team to be aesthetically unappealing and not effective in conveying the essence of the training. Ideas for color palette and content of the logo to be contextual relevant were discussed. A preliminary sketch was brainstormed, vetted by the collaborative research team for changes, and then refined for final use.

Video Testimonials

The collaborative research team agreed that real-life testimonials of loss, grief, and bereavement could be compelling and add depth to the CBL modules. Two CAB members facilitated an introduction to six RCs who expressed interest in participating in the video testimonials. Specific interview details including interview questions were distributed to the RCs willing to be interviewed (Table 1). Consents and photo releases were signed by each RC. They served in a consultative role and not treated as research participants. Each RC was compensated $200 for their single one-on-one interview, which lasted from 60–90 minutes. Filming of the RCs occurred in their local city in a conference meeting room. Prior to the interview, the interviewer established basic guidelines to outline expectations of both the interviewer and RC. A documentary interview style was implemented to keep the focus on the RC. The interviewer remained off camera and later the interviewer’s voice and questions were edited from the final product. Over six hours of video content was transcribed and reviewed by the collaborative research team. Segments of the interviews were categorized under three main themes; loss, grief, and bereavement. Each member of the collaborative research team reviewed the transcripts and two university team members selected an array of the most powerful excerpts. Final excerpts were arrived at through team consensus.

Table 1:

Interview Guide

Please tell me as much about your incarceration as you are comfortable sharing.

1. What’s it like to age in prison?

Can you tell me a story about aging in prison—especially challenges, issues, or concerns?

a. Impacted by aging and chronic illness

b. Treated differently in prison compared to community setting

c. What care or support is needed?

d. What could be done to improve the treatment of older inmates?

e. How do others respond to an aged or impaired inmate?

2. What losses are experienced during incarceration?

Can you tell me a story of living through a loss?

a. How are these losses handled?

b. What helped (you) get through that time?

c. What could be done to improve support when dealing with a loss?

3. What is it like to deal with death (of family or other inmates) while in prison?

Tell me a story about an experience you had with a death or grief while you were in prison.

a. Grieving process

b. Support was available

c. Suggested improvements

4. This project has three core values: Respect, Dignity, and Hope.

When thinking about aging and dying in prison, what do those values mean to you?

a. How could someone demonstrate respect, dignity, and hope to an individual

Respect is a sense or worth of value of a person. Respect means showing individuals consideration and treating them with courtesy and kindness.

Dignity is a sense of self-respect. Dignity implies that individuals have an innate right to respectful and ethical treatment.

Hope is a sense of confidence about the unfamiliar. Hope promotes reassurance that supportive care needs will be met with respect and dignity.

Graphic Novels

The university team collected images of a variety of adult faces that were diverse in race, ethnicity, age, size, and gender. The graphic artist used these to inspire an array of sketches depicting three options for each of a variety of interdisciplinary staff members along with incarcerated people. The collaborative research team came to a consensus on how each role would be depicted. In at least one instance a composite of two choices were merged to create the desired look. First round drafts were unveiled at a combined EAB/CAB meeting. Feedback informed further refinement in preparation for next steps (e.g., small-scale face-to-face usability testing).

Results and Outcomes

The content was refined in an iterative process. Shared decision-making and ongoing refinement of the training modules were collaboratively accomplished between the university and multimedia teams resulting in finalized content (Loeb, Penrod et al., 2017; Kitt-Lewis, Loeb, Myers et al., 2019).

Computer-Based Learning Program

Based on the final module content, the university team developed objectives, assessments, and further detailed content to promote congruence across the various components of the training product. Features to promote visual appeal, interaction, and user engagement were thoughtfully added to the CBL modules.

Objectives and Assessments

A consensus was reached on 20 objectives to be applied across six modules (module 1, four objectives; module 2A, four objectives; module 2B, three objectives; module 3, two objectives; module 4, four objectives; and module 5, three objectives). Each objective had 2 to 3 assessment questions, yielding 56 questions for all modules.

Images

The images received both negative (e.g., image looks staged; bars on cells are a thing of the past) and positive feedback (e.g., has a correctional feel; good depiction of “the yard”). Any image that received a “not at all” response was eliminated from further consideration. Images that received >50% of responses indicating “very well” were retained.

While some photographers/artists were easy to locate and responded to the team’s initial outreach, several challenges occurred when trying to locate and secure rights to the selected images. Some of the barriers included establishing ownership of the images, locating and reaching the photographer or artist, prioritizing images given the budgetary constraints, and obtaining appropriate permissions. In the end, ownership was determined for most of the images; however, a few images had to be abandoned, as no one would assert that they were the owner of the image.

Many photographers/artists did not respond to email or telephone outreach. The university team was unable to establish whether the contact information was no longer valid or if the artist opted not to respond due to disinterest. The artists who did communicate with the researchers either expressed interest in the project and provided a price per image or were not receptive to the idea and declined.

The price per image ranged from free to $4,000. Criteria for final photo selection included spending limits based on the quality of image, potential for multiple applications within the CBL training, and the general budget. Barriers continued to emerge. For example, even among the selected photos where the team had communicated with an identified photographer/artist—there were instances where despite several attempts to secure an appropriate contract, some photographers/artists never returned a signed contract.

Logo

The logo (Figure 3) was well received by the CAB members. Examples of positive feedback were an appreciation for the compassion and strength that were conveyed by the hands, relevance of the core values, and the color palette used in the background. Only one negative comment was received, and the CAB commented that someone that is not familiar with the ECAD-P acronym could be confused.

Figure 3:

Logo

Video Testimonials

Over six hours of video of RCs’ testimonials were edited. Short, yet powerful vignettes that represented each of the categorical themes were incorporated into Module 3: Loss and Grief. This was determined by the consensus of the university team to included three RCs’ testimonials for each theme.

Graphic Novels

The university team reviewed the graphic artist’s sketches and made suggestions for modification. The graphic artist revised the characters and graphic sketches based on the collaborative research team, CAB, and EAB suggestions. Upon team consensus, the final graphic images were approved.

In summary, the CAB reviewed prototypes of the modules with the new logo, as well as the purchased images, graphic novel sketches, and video testimonials, and provided thoughtful insights on the components of ECAD-P (e.g., need to enhance diversity of characters in graphic novels in terms of body size/shape, age, skin tone, and gender; need to humanize characters and be alert for looking unnecessarily angry or unfriendly). In addition, all elements were programmed into the CBL modules (Program Activities). The final version of ECAD-P was then ready for small-scale usability testing.

Programming

ECAD-P is built on a full stack web application using Hypertext Markup Language (HTML)/JavaScript as its main interface. HTML is the language used to create web pages and web applications. Look and feel was designed with Adobe Illustrator and developed using HTML elements. The minimum device software and hardware requirements for both tablet and desktop computers were ascertained. The requirements were: for tablets, iOS 6 or higher; Android 4.1 or higher, and Chrome mobile 36 or higher; for desktops, Chrome Browser (25 or higher); Safari (5 or higher); Internet Explorer (9 or higher); Microsoft Edge (12 or higher); and Firefox (20 or higher).

Research staff and multimedia experts developed methods to design learning activities in alignment with learning objectives (Gustafson & Branch, 2001; Peterson, 2003). Specifically, the ADDIE model (Dick, Care, & Carey, 2000; Mruganantham, 2015), an instructional systems design framework, was used to guide content development (see Table, Supplemental Digital Content 1). The ADDIE Model is a well-established approach for developing educational technology (e.g., e-learning) content. This model links instructional strategies to human learning theories, and helps guide the translational paper-based learning approaches into a technological format. Concepts were finalized for programming after multiple rounds of revisions. Reading level of all content was determined. Flesch-Kinkaid readability estimates for ECAD-P content ranged from Grade 6.3 (Module 5 “Innovative Strategies in Action”) to Grade 9.9 (Module 2B “Supportive Care – Medical Professionals”).

Content was developed with the goal of 30 minutes “seat-time” per module, with a total training time of 2.5 hours. Testing of seat time was confirmed via alpha testing by the university research team, technology company team, and CAB—their educational levels and disciplinary/vocational backgrounds were diverse. In addition to establishing the seat time of the program, this testing served as an in-house beta test for the web platform’s stability and to identify code errors. AGILE programming (Dattatreya, Rao, & Rayudu, 2012; Khalid, Zahra, & Khan, 2014), a collaborative and incremental programming methodology which incorporates and iterative design process, was utilized and revisions were programmed and re-tested for stability by the research team and technology company on an on-going basis. The program flow can be found in Figure 4.

Figure 4:

ECAD-P Programming Flowsheet

The programming flowsheet provides a map of how a user would progress through the CBL module. A key is provided to explain the specific components of the flowsheet.

Discussion

Although healthcare staff members have the expertise and knowledge to provide health care to all incarcerated people, providing care for those who are chronically ill and dying requires supportive care approaches from the entire multidisciplinary staff (Loeb, Wion, Penrod, et al., 2018; Kitt-Lewis, Loeb et al., 2019). Corrections officers, unit managers, and others have more day-to-day contact with incarcerated people than do healthcare personnel. Therefore, they are more likely to notice indicators of declining health in the incarcerated people. Evidenced-based training products to prepare all correctional personnel, not just correctional health care workers, on the care needs of incarcerated people is critical.

To comply with accreditation standards, corrections facilities are responsible for preparing personnel who are knowledgeable in all aspects of corrections and understand the population in their custody and control (American Correctional Association, n.d.; NCCHC, n.d.). To provide the educational needs at a minimal cost, correctional settings rely on “homegrown” educational products (Loeb, Penrod, Myers et al., 2017). However, high-quality developed training products that were free or offered at a minimal cost were highly desired. The rigorous, evidence-based, and collaborative methods used in the development of ECAD-P ensures that it is a legitimate, acceptable, and sustainable training product for correctional settings. This aligns with the recommendations of Stevens et al. (2018) that stressed the importance of designing programmatic initiatives that will not only be endorsed but enduring. Correctional systems and their employees are responsible for ensuring not only custody and control, but also for providing care. CBL products can deliver up-to-date evidence-based training to support corrections personnel, which can build confidence and improve morale (Ridley-Turner, 2002; Garland, 2002). Training correctional staff on relevant issues when caring for these special populations may lead to a positive shift in culture and uptake of enhanced care for aged and dying incarcerated people. ECAD-P is a technology-based learning system that can contribute to filling the gap in training aimed at improving care for older and dying incarcerated people.

Implementing this type of education for all corrections personnel is a novel approach and the collaborative research team having these established relationships with corrections insiders provided a powerful lens into these complex systems (Langley, Moen, Lolan, Nolan, Morman, & Provost, 2009). Each CAB member played a valuable role in the iterative process as we developed evidence-based, media rich, and interactive CBL modules. In addition, the CAB members were key in our ability to produce video testimonials of RCs’ experiences with loss, grief, and bereavement in prison. Impactful excerpts are embedded in the program and add value to the training. The collaborative research team developed a contextually relevant product with great specificity that could not have been achieved without the CAB and EAB—a finding that is congruent with Apa and colleagues (2012) emphasis on the importance of engaging with an Advisory Council in order to achieve mutual goals when undertaking research in prisons. In addition to the development process, the CAB and EAB guided the team as we attempted to build relationships with state departments of corrections and county jails. The EAB and CAB made suggestions on how to gain entrée into the “system” and how to overcome challenges and barriers that we face as we move into usability testing of the ECAD-P training modules. ECAD-P was programmed using state-of-the-art approaches, which provide an engaging and friendly user interface while simultaneously maintaining strict cybersecurity. The iterative and agile programming approach provides a media rich experience for the user and assures security of data and content that can be implemented within the walls of correctional facilities. Further examination of developing and testing the scalable unit is needed prior to full-scale dissemination (Barker et al., 2015). The importance of including qualitative/open-ended questions when evaluating training must not be overlooked. Key questions to consider are: inquiring what users liked most and least about a product; eliciting suggestions for change and conversely what should not be changed; and ascertaining how a training product was helpful and/or unhelpful?

The funding for this study to develop computer-based training was funded by Small Business Technology Transfer Grants (i.e., STTR R41 and R42). The purpose behind these grant mechanisms that fund this line of research are for small businesses to partner with nonprofit research institutions to conduct innovative research and development for products with commercial potential. Since it would not be feasible to maintain a current, fully functioning technology product without financial resources, potential future commercial partners are beginning to be explored. However, at this point in time, it is impossible to predict whether or not our product will be sold and profitable.

Conclusions

Completion of the training product (ECAD-P) advances the research and development necessary to further the goal of full-scale dissemination. ECAD-P contains six modules, created under the careful guidance of the university team, the CAB, and the EAB, an approach which is congruent with Stevens and colleagues’ (2018) suggestion to involve experts in the content area and contextual setting. Next steps include developing the scalable unit. In-person small-scale usability testing with approximately 20 participants in a large inner city jail and a large state DOC to ensure racial/ethnic diversity and a variety of disciplinary roles (e.g., nursing, security, counseling, and chaplaincy) to ensure that both men and women are represented. Research suggests that between 5 and 10 usability testers are needed to identify the majority of usability issues (Earnshaw, Tawfik, & Schmidt, 2018; Wolpin & Stewart, 2011). Usability testing with 20 participants will assure not only identification of programs which may impede successful use of the program, but will also provide a high level of specificity for user preferences which can inform the program’s potential for sustained update. Early identification of usability concerns and programming errors is critical to successful dissemination. Small-scale usability testing will identify tasks and programming issues that create a barrier to successful completion of the program. Usability outcomes regarding program functionality as well as feedback from end users are critical to presenting a polished, refined program for large scale dissemination. These preliminary findings will allow the research team to make needed refinements and revisions prior to testing the product in the field, without the direct, on-site involvement of the research team. The research team will conduct large-scale usability testing with corrections personnel from diverse correctional settings in the United States. Expanded testing of the CBL system is a necessary investment prior to considering commercialization and mass dissemination.

Additional research needs to be conducted on the effectiveness of the pre/post-test assessments. Through the expertise of the research team, EAB, and CAB, content and face validity were established in the development phase of the product. However, research will be needed to test reliability and validity of the pre/post-tests. Establishing a reliable and valid pre/post-test assessment will ensure that the educational modules meet the intended outcomes of the training product. The impact of the training will be an important long-range goal to assess effectiveness. For example, a longitudinal examination of the impact of the training on care of incarcerated people (e.g., comparative effectiveness trial, implementation science approach).

Developing an evidence-based CBL product is dynamic and time-intensive. That is why researchers in this field need to be vigilant in sharing lessons learned. Much of the training materials used in correctional settings are developed “in house”. Therefore, the strategies described herein can be applied by forensic nurses in correctional settings for whom training is a responsibility, as well as by the broader array of correctional training officers, whose goal is to develop CBL programs that are usable and relevant to their environment. In addition, lessons learned may be applied to other complex organizations with restrictive environments (e.g., forensic mental health facilities). This research team intends to apply the lessons learned in this project to design and develop a CBL program to train incarcerated people to serve as peer caregivers. The experience gained from creating the ECAD-P program prepares the team to be more efficient in the areas of developing content and producing module templates, as well as considering potential graphic images and estimating programming time. Opportunities exist to establish academic and correctional partnerships to create and share content to better prepare the workforce to serve our community both now and in the future.