Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jan 1.
Published in final edited form as: J Forensic Nurs. 2021 Jul-Sep;17(3):E24–E33. doi: 10.1097/JFN.0000000000000337

The Implementation of the Sexual Assault Forensic Examination Telehealth Center: A Program Evaluation

Sheridan Miyamoto 1, Elizabeth Thiede 1, Elizabeth N Wright 1, Diane Berish 1, Daniel F Perkins 2, Cynthia Bittner 1, Lorah Dorn 1, Dennis Scanlon 3
PMCID: PMC8387343  NIHMSID: NIHMS1696367  PMID: 34132652

Abstract

Introduction:

Substantial disparities in the quality of post-sexual assault (SA) care exist in the US, particularly in rural areas. This study evaluates the implementation of the Sexual Assault Forensic Examination Telehealth (SAFE-T) Center, a program to improve SA care by increasing access to experienced Sexual Assault Nurse Examiners (SANEs) via telehealth, in three rural hospitals.

Materials and Methods:

The Dynamic Sustainability Framework (DSF) guided the implementation of the intervention. Survey and implementation data were evaluated one year after implementation using a non-experimental pre-post design. Outcomes include patient and nurse perceptions of telehealth, local site nurse (LSN) confidence, and hospital protocol/policy changes.

Results:

Forty-one telehealth consultations were completed in the program’s first year. An average of 34 system-level protocol changes were made per site. LSNs demonstrated statistically significant increases in confidence to provide SA care at one year. LSNs and TeleSANEs (expert consultants) reported that quality of SA care improved (87% and 83%, respectively). Patients highly rated the care they received (83%), reported telehealth improved care (78%) and reported feeling better after the examination (74%).

Discussion:

Using the DSF framework for implementation supported a tailored approach, successful adoption, and allowed for program iteration based on lessons learned.

Conclusions:

The SAFE-T model resulted in improved local nurse confidence in provision of SA care, nurse perception of improvement in care quality, and high patient care experience ratings. These findings and the use of the DSF framework have implications for SA specialty care implementation in rural communities.

Keywords: Sexual Assault, Sexual Assault Nurse Examiner, Telehealth, Telemedicine, Forensic Nursing, Rural

Introduction

Patients who receive a sexual assault (SA) exam from a Sexual Assault Nurse Examiner (SANE), a nurse specially trained in the provision of patient-centered, trauma-informed forensic care, experience better health and prosecutorial outcomes compared to those who receive SA exams from untrained providers (Campbell, et al., 2012; Campbell et al., 2005). The United States has experienced persistent SANE shortages and, as a result, solutions are needed to ensure equitable access to the specialized healthcare that SANEs provide. Telehealth has become a well-accepted, effective resource for providing access to other specialty healthcare services such as psychiatry, radiology, infectious disease, and stroke in rural and underserved areas (Hoth et al., 2019; Price, et al., 2019; Schwamm et al., 2009; Weinstein et al., 2014).

Previous studies show that telehealth may be a solution to address healthcare inequities that limit access to SANE-led SA care in rural communities (Miyamoto et al., 2014). Rural residents comprise nearly 20% of the U.S. population and often experience disparate outcomes in an array of health indicators (Moy et al., 2017; United States Census Bureau, 2017). Difficulty obtaining specialty healthcare services can contribute to these disparate health outcomes. Telehealth removes barriers to accessing quality, specialty health care in rural communities by providing access to experts and decreasing the need for travel to obtain specialty care (Hassija & Gray, 2011; Pammer et al., 2001; Potter et al., 2014; Schwamm et al., 2009; Weinstein et al., 2014). Additionally, local economies benefit from healthcare spending that remains within the local community (Livingood et al., 2007).

The Sexual Assault Forensic Examination Telehealth Center

Provision of trauma-informed, person-centered care for victims of sexual assault requires specialist training. SANEs are registered nurses who have completed 40 hours of classroom training, followed by clinical training and mentoring. As with many health care specialties, there are not nearly enough SANE specialists to ensure coverage across all geographic areas in the U.S. and this gap is especially stark in rural areas (Thiede & Miyamoto, 2020; United States Government Accountability Office, 2016). Accessibility of specialty training, low retention of SANEs post-training, and lack of hospital administrative support for the development of SANE response teams have been cited as barriers to growing the SANE workforce (United States Government Accountability Office, 2016).

Recognizing that solutions are needed to overcome these unmet needs in SA care, the SAFE-T Center was created in 2017 with funding from the Department of Justice (DOJ), Office for Victims of Crime to enhance access to quality SA care in rural and underserved areas. The SAFE-T Center program provides live, 24/7 access to experienced, International Association of Forensic Nurse (IAFN)-certified SANEs via telehealth technology (TeleSANEs) to our partner hospitals, ongoing opportunities for education and training, secure telehealth technology that allows for SA examination quality assurance in real-time, and administrative support in navigating program and policy changes. Local site nurses (LSNs), who provide the SA care while the TeleSANE provides quality assurance remotely, complete a baseline of 40 hours of training in SA care. Additional clinical training is provided by the SAFE-T Center. When a SA examination is performed at one of our partner hospitals, the SAFE-T Center clinical intervention includes an expert TeleSANE who provides real-time support to both the LSN and the patient, ensuring best practices are followed, proper evidence collection techniques are used, and a supportive environment exists for the patient. In this paper, we describe our approach to the evaluation of the SAFE-T program using a continuous quality improvement (CQI)-based framework: The Dynamic Sustainability Framework (Chambers et al.,2013), and present preliminary evaluation outcomes from the first year of program delivery at three rural hospital sites.

Conceptual Framework

Continuous evaluation and CQI guide our framework of iterative refinement of the SAFE-T model. We are working to build a telehealth model for adaption in a variety of health systems and communities with differing resources and needs. As our model is placed in diverse settings, we must allow for the evolution of needs over time and embrace ongoing change and improvement. Program adaptation may lead to the generation of essential knowledge and considerations for others wanting to implement a telehealth solution for SA care.

We selected the Dynamic Sustainability Framework (DSF) as the guiding implementation and evaluation framework for our model (Chambers et al., 2013). The DSF challenges traditional, linear implementation frameworks by arguing that a program should not be optimized prior to the intervention as that limits real-time, agile adjustments, enhancements, and customization needed to implement effective, community-based interventions in varied settings (Chambers et al., 2013). Rather, achieving optimal fit within a practice setting and larger ecological system involves repeated evaluation of the intervention, practice setting, and larger ecological system (Figure 1).

Figure 1:

Figure 1:

SAFE-T adaptation of the Dynamic Sustainability Framework for evaluation

Note: Figure adapted from Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The Dynamic Sustainability Framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117. doi:10.1186/1748–5908-8–117

The DSF consists of three major constructs: intervention, practice setting, and the larger ecological system (Chambers et al., 2013). The intervention construct includes components of the intervention, the delivery platform, practitioners delivering the intervention, and outcomes of the intervention. The practice setting construct includes staffing issues, informational systems, organizational culture, business models, training, and supervision. The ecological system includes various practice settings implementing the intervention, external policies and regulations, and broader population characteristics. The ecological system is critical to successful and sustainable intervention as the practice setting and the intervention are influenced by external systems. The DSF guides how we measure and evaluate the SAFE-T Center’s impact over time.

The implementation, evaluation, and dissemination of outcomes from the SAFE-T Center’s community-based, multisite intervention is multifaceted and involves a long-term agenda. This paper serves to discuss early implementation, data collection and results of the measurement constructs we created to understand program impact from the perspective of the health system, nurse participants and patients. In line with the iterative spirit of the DSF framework, we share lessons learned and how those lessons will influence future program implementation. Representative of the long-term agenda to fully evaluate SAFE-T program outcomes, we have other evaluation components, including qualitative evaluation, which are not presented here.

We hypothesized that the implementation of the SAFE-T program across the partner sites would: 1) result in changes in system-level policies to reflect evidence-based sexual assault care; 2) lead to increases in nurse confidence in providing SA care; and 3) result in high patient care experience ratings. The purpose of this paper is to examine these hypotheses using year-one outcomes through the lens of the intervention (SAFE-T clinical intervention) and practice (SAFE-T Center program) constructs of the DSF and to report preliminary findings.

Methods

Study design and procedure

This study design is a non-experimental, pre-post design without a concurrent control group. Data were collected over 27 months from June 2017 to Fall 2019, from initial (baseline) evaluations of site partners through the first year of implementation at three partner hospitals.

Rural partner sites

Partner site hospitals were volunteers that were non-randomly selected based on the following criteria: lack of comprehensive SANE response to SA, strong hospital administrative support to implement a SANE team and telehealth program, and rural status. Rural status was determined at the county-level, using the Center for Rural Pennsylvania’s rurality criteria (The Center for Rural Pennsylvania website). Our partner hospitals varied in size with 25, 71, and 211 in-patient beds. One site was designated as a Critical Access Hospital, characterized by having 25 or fewer acute inpatient beds and located greater than 35 miles from another hospital (Pennsylvania Office of Rural Health website). All hospitals reported having at least 10 SA cases in the prior year. Although each site had 3–4 nurses with some prior SANE training, there was substantial variation in the time since training occurred (1–6 years) and most had no continuing education beyond initial training. A planned fourth site withdrew prior to implementation as the hospital’s larger healthcare system decided to create an internal, system-wide SA response program.

Participants and recruitment

Clinician participants were nurses engaged in SAFE-T consultations including: 12 LSNs and 13 IAFN-certified adolescent/adult SANEs hired as expert consultant TeleSANEs by SAFE-T to provide 24/7, real-time mentoring and support to the LSNs during SA case examinations.

Patients were recruited from partner hospitals when presenting for a SA exam. They were offered a telehealth consultation by the LSN, and, if accepted, the patient was offered the opportunity to participate in research. When the partner hospital requested a SA consultation, the SAFE-T Center was notified and a TeleSANE logged into the telehealth platform to connect with the LSN. After consenting to a SA exam, the LSN gave the patient a secure iPad that played a video explaining the purpose of the study. If after watching the video, the patient expressed interest in participation, TeleSANEs explained the study in detail, answered study-related questions, and consented patients. Patients were not eligible for study participation if they were under 12 years of age, non-English speaking, currently incarcerated, or unable to give informed consent. Parent/guardians present with a minor could consent on the minor’s behalf if the minor also provided assent for the study. Minors without a parent present were able to provide assent. Patients could consent to telehealth and elect not to participate in research without any penalty or change in care. Prisoners were eligible for telehealth, but not research.

Measures

We evaluated the SAFE-T program at year-one by focusing on the intervention and practice setting domains of the DSF (Figure 1). To evaluate the intervention domain, surveys of LSNs, TeleSANEs, and patients were conducted at various timepoints (see Supplemental Digital Content 1). Following 40-hour SANE training and prior to conducting any examinations with SAFE-T, LSNs were given baseline (T0) surveys asking them to rate their confidence in conducting various aspects of a SA examination. LSN confidence was measured again one year after the SAFE-T program was in place (T1) to determine what, if any, effect the SAFE-T intervention had on LSN practice confidence. Additionally, patients, LSNs and TeleSANEs were asked to assess their experience after each telehealth consultation. Each survey was developed for the project and used Likert scale responses.

To evaluate the impact of the SAFE-T program on the practice setting in the first year, baseline surveys were completed at each partner site that provided information on the hospitals protocols, policies, and clinical interventions regarding forensic examinations. Throughout the first year, researchers collected data on the system-level protocol and policy changes from baseline that developed and strengthened the SANE workforce within their institutions and were changed or were implemented in year-one as a result of the SAFE-T program. Changes across policies, education/training, exam process, SANE team, and photography/documentation were tracked. Examples include whether hospitals transitioned from a physician-led SA forensic examination to a SANE-led (LSNs) exam, whether LSNs received protected time to complete continuing education and attend peer-review meetings, use of updated SA collection kits, number the nurses recruited to join the SANE team, and whether the site updated equipment.

Data collection

Hospital-level data were collected via research team observations continuously throughout the study period. Individual-level survey data were collected at various timepoints (Supplemental Digital Content 1) from hospital administrators, LSNs, TeleSANEs and patients. Survey data was collected electronically using REDCap (a clinical research software platform) (Harris et al., 2009; 2019) or paper forms when electronic access was not feasible, such as in large group meetings. Researchers recorded the number of telehealth interventions refused, the number of consultations not conducted due to technology failure, and LSN retention in the intervention (i.e., local SA nurse turnover). Research procedures were approved by the university’s Institutional Review Board (IRB) and informed consent for research was obtained from all participants.

Analysis Plan

All observational data were entered by research staff and survey responses were collected and managed using REDCap electronic data capture tools at The Pennsylvania State University (Harris et al., 2019; Harris et al., 2009). Statistical analyses were generated using SAS© software Version [9.4] (SAS Institute Inc, 2014). All data were examined for descriptive information (e.g., means for continuous variables and frequencies for categorical variables) and missingness. Missing data were due to unanswered survey questions by some participants and those questions were dropped from the analysis with sample size changes noted within the tables. Data were aggregated across participant group and examined for trends. For the subgroup of eight LSNs who completed a year-one follow-up set of surveys, T0 to T1 change scores were calculated (T1 – T0) using a Chi-square test.

Results

In the first year of the program there were 52 total calls to SAFE-T from three partner hospitals, resulting in 43 SA examinations, eight advice-only calls that did not result in a telehealth consultation, and one patient who left before receiving an examination. Of patients eligible for a telehealth consultation, three (7.0%) declined the service. Of the 40 telehealth consultations conducted, nine (22.5%) patients did not meet research eligibility criteria (e.g., non-English speaking or incarcerated), leaving 31 patients who were eligible for research recruitment. Of those, five patients (16.1%) declined to participate in research, two left without completing a survey (6.4%), and one was excluded due to an incomplete consent form (3.2%). Thus, 23 (74.2%) eligible research participants were included in the analysis (see Supplemental Digital Content 2). Table 1 displays the demographic characteristics for all SA patients who received a telehealth consultation.

Table 1.

Demographic characteristics for patients who received a sexual assault telehealth consultation (n=40)

Age (years), n (%)

 12–17 8 (20.0)
 18–24 13 (32.5)
 25–29 7 (17.5)
 30–39 6 (15.0)
 40–49 2 (5.0)
 50–59 4 (10.0)

Gender Identification(s), n (%)

 Female 31 (77.5)
 Male 7 (17.5)
 Non-binary 2 (5.0)

Race, n (%)

 White 20 (50.0)
 Black or African American 4 (10.0)
 Prefer not to answer 3 (7.5)
 Mis sing 13 (32.5)

Incarcerated, n (%) 32 (80.0)

Patient Care Experience

Table 2 displays results for the patient care experience surveys (n=23) completed after each telehealth case consultation. The majority stated that having a telehealth consultation improved the care they received (Agree or Strongly Agree = 78%) and that the examination helped them to feel better (Agree or Strongly Agree = 74%). Moreover, most patients rated the care they received as ‘excellent’ or ‘very good’ (83%).

Table 2.

Patient post-telehealth consultation care perspective ratings (n=23)

Question: Having a telehealth consultation improved the care I received today, %

 Strongly disagree 0
 Disagree 9
 Somewhat disagree 4
 Somewhat agree 9
 Agree 26
 Strongly Agree 52

Question: The examination helped me to feel better, %

 Strongly disagree 4
 Disagree 0
 Somewhat disagree 13
 Somewhat agree 9
 Agree 35
 Strongly agree 39

Question: Please rate the care you received today, %

 Very Poor 0
 Poor 0
 Fair 4
 Good 13
 Very Good 13
 Excellent 70

Note: Of the n=40 patients who received telehealth examination, n=23 patients completed a patient care perspective survey

LSN Confidence in Provision of SA Care

At the start of the program, there were 12 LSNs providing SA care in partner site hospitals. Shortly after the SAFE-T program was implemented, three LSNs left their positions, with each conducting two or fewer SA examinations. A fourth LSN did not complete the follow-up survey. At baseline, 50% of the original 12 LSNs reported feeling “very” or “extremely” confident in their ability to provide compassionate support to SA victims (Table 3). However, LSNs’ confidence in their ability to perform a complete forensic examination and self-rated confidence in their ability to capture quality images was lower. At year-one follow-up, LSN confidence significantly increased for all aspects of SA care: Ability to provide compassionate support (p < .0001); Perform a complete forensic examination (p =0.027); and Ability to capture quality images (p = .016) (Table 3).

Table 3.

Local site nurse (LSN) change in confidence from baseline (T0) to 1-year evaluation (T1)

graphic file with name nihms-1696367-t0002.jpg

Note: Question wording: “How confident are you in your ability to provide compassionate support?”; “How confident are you in your ability to perform a complete forensic exam from beginning to end?”; and “How confident are you in your ability to capture quality images/pictures?”

LSN and TeleSANE Post-Consultation Results

Three LSN post-consultation surveys and two TeleSANE post-consultation surveys were not completed, leaving 37 LSN and 38 TeleSANE post-consultation surveys available for analysis. The LSNs provided feedback on 37 of the 40 consultations (Table 4), reporting high levels of satisfaction (‘Satisfied’ or ‘Very Satisfied’) with the overall experience with the telehealth technology (94.4%), high satisfaction with reliability of the telehealth equipment (81.1%), and high levels of support (‘Supported’ or ‘Very Supported’) from the TeleSANE consultant (89.2%). The vast majority endorsed that having a telehealth consultation was ‘Important’ or ‘Very Important’ (83.8%) and that the consultation improved the quality of the examination (83.3%).

Table 4.

Local site nurse (LSN) post-telehealth consultation perspective ratings (n=37)

Question: Overall experience with technology (n=36), %

Very dissatisfied 0
Dissatisfied 0
Somewhat dissatisfied 2.8
Somewhat satisfied 2.8
Satisfied 19.4
Very Satisfied 75.0

Question: Reliability of telehealth equipment, %

Very dissatisfied 0
Dissatisfied 0
Somewhat dissatisfied 5.4
Somewhat satisfied 13.5
Satisfied 18.9
Very Satisfied 62.2

Question: How well supported did you feel, %

Not at all 2.7
Slightly 2.7
Moderately supported 5.4
Supported 5.4
Very supported 83.8

Question: How important was access to a telehealth consultation, %

Not at all 2.7
Slightly 2.7
Moderately important 10.8
Important 16.2
Very important 67.6

Question: How much did telehealth improve the quality of the examination (n=36), %

Not at all 2.8
Slightly 5.6
Moderately improved 8.3
Improved 25
Very improved 58.3

The expert consultant TeleSANEs provided feedback on 38 of the 40 consultations (Table 5). TeleSANEs rated telehealth consultations as ‘Very’ or ‘Extremely Important’ (89.4%) and endorsed that telehealth was important in improving examination quality (‘Very’ or ‘Extremely Important’ = 89.5%), yet they gave the overall effectiveness of the consultation somewhat lower ratings (‘High’ or ‘Very High’ = 68.5%).

Table 5.

TeleSANE post-telehealth consultation perspective ratings (n=38)

Question: How important was it that this examination included a telehealth consultation, %

Not at all 0
Slightly 2.6
Moderately 7.9
Very 52.6
Extremely 36.8

Question: How important was telehealth to improving the quality of the examination, %

Not at all 0
Slightly 2.6
Moderately 7.9
Very 57.9
Extremely 31.6

Question: Effectiveness of the consultation, %

Very low 2.6
Low 0
Somewhat low 2.6
Somewhat high 26.3
High 39.5
Very high 29.0

System-Level Outcomes

Examples of system-level changes across the partner sites as a result of the SAFE-T Center program implementation are displayed in Supplemental Digital Content 3. These results are categorized according to their respective DSF domains (intervention and practice setting). Partner hospital assessments demonstrated an average of 34 system-level changes (e.g. policy, procedure, and protocol changes) per site resulting from the implementation of the SAFE-T partnership. An average of 10 policy/procedure changes, 10 education-related changes, 7 exam-related process changes, 4 SANE team-related changes, and 3 photography/documentation changes occurred at each site. Additionally, all three partner sites established a paid on-call system and recruited a team with a minimum of four LSNs, all LSNs received training in SA forensic examination, two of the three partner sites transitioned from physician-led to a completely SANE-led SA forensic examination and victim advocates, community-based professionals trained to provide victims with support and resources, responded to the hospital to offer services to the patient for every examination.

Discussion

This study evaluated the implementation and early program outcomes of a comprehensive expert SA telehealth care model in three rural community hospitals using the Dynamic Sustainability Framework. The SAFE-T model provided SANE program development and guidance, training for local SA nurse teams, and access to experienced TeleSANEs for live examination consultation, mentoring, and quality assurance via secure telehealth technology.

LSN Confidence

Following the first year of SAFE-T implementation, LSNs confidence in their ability to provide compassionate support, perform a complete forensic examination, and capture quality images during sexual assault examinations was significantly higher than at baseline; findings that may positively influence both the quality of care provided and SANE retention. Increased confidence may stem from the support provided by a more experienced TeleSANE partner who acts as a sounding board for decision-making, models best practices and trauma-informed, person-centered interactions that the LSN can adopt in their own practice, and provides quality assurance through case review.

LSN Retention

Self-confidence, quality mentoring, and job satisfaction are key to retention of nurses. A primary goal of this model is to increase the availability of SANE trained nurses in rural areas and to retain this resource to sustain quality local response to SA (Buffington et al., 2012; Cowin & Hengstberger-Sims, 2006). The vast majority of SANE-trained nurses will leave the specialty within the first two years due to lack of support, dissatisfaction with compensation and the emotional demands of the role (United States Government Accountability Office, 2016). With the SAFE-T model, 75% of LSNs were still actively providing SA care in their community one year after program implementation. High retention rates may be attributed to increased local hospital administrative and community support and inclusion in a community network of SANEs (site nurses and TeleSANEs) that interact through consultations, peer review, and educational trainings.

Quality of Care

Site changes in protocols, LSN satisfaction with telehealth support, and increased LSN confidence in their ability to provide SA care have important implications for the quality of care provided. Increased LSN confidence and support from a TeleSANE likely translates to increased quality of care. A previous study of forensic SA telehealth consultation demonstrated improved examination quality in sites with expert telehealth consultation versus similar hospitals without telehealth consultation (Miyamoto et al., 2014). The SAFE-T telehealth model has many parallels to the Project ECHO model, which has been shown to improve patient care in areas with limited access to specialists by supporting primary care providers to manage complex health problems through specialist telehealth consultation and co-management (Anderson et al., 2017; Zhou et al., 2016). A Project Echo systematic review reported local provider satisfaction, increased knowledge and increased clinical confidence (McBain et al., 2019).

Another example of a system-level change with strong potential to impact quality (Campbell, 2006; Patterson & Tringali, 2015) was that partner hospitals shifted protocols from verbally offering a patient the choice to have an advocate called to proactively requesting advocates to respond to the hospital in person. This change allows advocates to share their purpose and role with the patient directly, allowing for an informed decision by the patient. This change resulted in every SA patient choosing to have an advocate present during their examination once SAFE-T was implemented.

The overwhelming majority of both LSNs and TeleSANEs endorsed that telehealth consultations were important and improved the quality of examination afforded to SA patients. Our findings are similar to prior studies evaluating SA telehealth impact on local nurses’ perceptions of program value (MacLeod et al., 2009; Walsh et al., 2019).

Patient Outcomes

Nearly all patients presenting for SA care (93%) consented to expert telehealth consultation, a strong endorsement of acceptability. Ours is the first study of SA telehealth delivery to explore patient perceptions of the care they received. An important, and to our knowledge not previously identified finding, was high patient endorsement that the examination helped them feel better (74%). We believe this highlights that when experienced SANEs partner with less-experienced LSNs to deliver patient-centered, trauma-informed, evidence-based care, it can be the first step in healing for a survivor of SA.

System-level Lessons

We found the SAFE-T model resulted in numerous system-level changes. Partner site changes indicate that the SAFE-T model positively impacted several clinical aspects of SA care provision. Site changes related to establishing and maintaining SANE teams, such as implementing on-call pay or dedicated time for continuing education, reflect sites’ program buy-in as these changes require site financial commitment.

The willingness of sites to implement the volume and array of changes likely relates to both the criteria used for selecting partner sites as well as our dynamic, adaptable approach to implementing the SAFE-T model in each unique setting. We specifically selected partner sites that showed strong administrative support for building and maintaining a team of SANEs which likely contributed substantially to implementation success.

All sites were effectively able to partner with us to establish and support a comprehensive SANE response in their community where it didn’t exist before. Given that sites had prior failed attempts to build a comprehensive team, we believe there were key system changes that were essential to this success. Those elements are paid time for nurses to attend training, establishment of a paid on-call structure, and administrator commitment to reviewing program outcomes, so they stay connected to program value. Given the recognition of these keys to success, we have adapted our model going forward to include that site selection should be based on site commitment to a ‘hospital partnership model’ in which hospitals commit resources to develop and support a coordinated, paid team of SANEs and engage in sustainability planning for telehealth support with the SAFE-T Center.

Measurement and Evaluation Lessons

We gained key insights from early implementation of our evaluation measures. First, some of our initial units of measure were insufficient to generate meaningful outcomes. For example, reporting the number of policy or protocol changes (Supplemental Digital Content 1), give no indication of how substantial or impactful those changes may be. Yet, when they are described (refer to Supplemental Digital Content 3), we can begin to see clear clinical impact and that some have more impact than others. We will work to evolve our measurement models to better measure impact of changes made.

A second lesson is that community-based program implementation takes extraordinary effort and time. While that investment is crucial, and we believe core to the success of this program, it hinders our ability to rapidly report on findings. Further, this program was implemented in rural, low-population density sites with substantial deficits in ability to provide quality SANE care. This means that while services were put in place in areas of great need, those sites are slow to generate cases. One year is not enough time to generate the number of cases needed for robust program evaluation. We offer these preliminary findings to share early program potential with the caveat that we must reevaluate outcomes from diverse community sites and with greater numbers of cases and participants.

Limitations

There are several limitations to this study. While we were able to show successful implementation in three communities due to our flexible, DSF-driven implementation framework, sites were purposively selected for their motivation to improve local SA care delivery. Therefore, implementation success may differ in other settings. While a fourth site dropped out just before program launch, we view this as a success as the parent healthcare organization opted to build their own comprehensive SANE response to benefit their system and in turn, the SA survivor population. Finally, these pilot programs were launched in low-population density, rural communities, resulting in a low volume of SA patients. Our findings represent early trends which should be validated and reported as more cases are completed over time. Additionally, outcomes are not compared to hospitals without access to telehealth programs which would provide an enhanced understanding of program impact.

Future evaluation will include additional sites as SAFE-T expands as well as comparison to control site hospitals that have an established SA response, but do not have access to telehealth consultation to allow for a rigorous assessment of the impact of telehealth on the quality of SA exam and patient care experience. With recent approval from our university’s Institutional Review Board, future studies will also include the recruitment and evaluation of state prisoners receiving SA exams.

Conclusions

Our preliminary findings suggest that a comprehensive, nurse-led SA telehealth program that provides administrative guidance for local team development, education and ongoing training for local teams, and live peer-review and quality assurance via telehealth is feasible, acceptable, and that local nurses, consultant nurses, and patients viewed the program as having a positive impact on SA care delivery in three rural hospitals. Findings one-year after program implementation support that the SAFE-T program stimulated the number of SANE-trained nurses working in underserved areas, improved local nurses’ confidence to provide quality forensic care, and resulted in high levels of program satisfaction for both nurses and patients. The use of the DSF framework to community-based implementation of a specialty forensic telehealth model and lessons learned in the first year of implementation may have implications for the implementation of specialty care telehealth models of care in rural communities.

Supplementary Material

Supplemental Data_2
Supplemental Data_3
Supplemental Data_1

Acknowledgements

The authors acknowledge Regina Lozinski, Joslyn Neiderer, and Madisyn Barnes for their thoughtful editing and preparation of this manuscript.

Funding

Research reported in this publication was supported by the United States Department of Justice (DOJ), Office for Victims of Crime (OVC) Award #: 2016-NE-BX-K001, National Institutes of Health, Eunice Kennedy Shriver National Institute of Child Health and Human Development under Award Number P50HD089922 and by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1 TR002014. The funding sources were not involved in the study design and the collection, analysis and interpretation of the data, writing the report, nor in the decision to submit the paper for publication. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication. The content is solely the responsibility of the authors and does not necessarily represent the official views of the U.S. DOJ or National Institutes of Health.

Footnotes

Author Disclosure Statement

The authors declare that no competing financial interests exist.

References

  1. Anderson D, Zlateva I, Davis B, Bifulco L, Giannotti T, Coman E, & Spegman D (2017). Improving pain care with project ECHO in community health centers. Pain Medicine, 18(10), 1882–1889. doi: 10.1093/pm/pnx187 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Buffington A, Zwink J, Fink R, DeVine D, & Sanders C (2012). Factors affecting nurse retention at an academic Magnet® hospital. JONA: The Journal of Nursing Administration, 42(5), 273–281. doi: 10.1097/NNA.0b013e3182433812 [DOI] [PubMed] [Google Scholar]
  3. Campbell R (2006). Rape survivors’ experiences with the legal and medical systems: Do rape victim advocates make a difference? Violence Against Women, 12(1), 30–45. doi: 10.1177/1077801205277539 [DOI] [PubMed] [Google Scholar]
  4. Campbell R, Bybee D, Kelley KD, Dworkin ER, & Patterson D (2012). The impact of sexual assault nurse examiner (SANE) program services on law enforcement investigational practices: A mediational analysis. Criminal Justice and Behavior, 39(2), 169–184. doi: 10.1177/0093854811428038 [DOI] [Google Scholar]
  5. Campbell R, Patterson D, & Lichty LF (2005). The effectiveness of sexual assault nurse examiner (SANE) programs: A review of psychological, medical, legal, and community outcomes. Trauma, Violence, Abuse, 6(4), 313–329. doi: 10.1177/1524838005280328 [DOI] [PubMed] [Google Scholar]
  6. Chambers DA, Glasgow RE, & Stange KC (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117. doi: 10.1186/1748-5908-8-117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cowin LS, & Hengstberger-Sims C (2006). New graduate nurse self-concept and retention: A longitudinal survey. International Journal of Nursing Studies, 43(1), 59–70. doi: 10.1016/j.ijnurstu.2005.03.004 [DOI] [PubMed] [Google Scholar]
  8. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, McLeod L, Delacqua G, Delacqua F, & Kirby J (2019). The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics, 95, 103208. doi: 10.1016/j.jbi.2019.103208 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hassija C, & Gray MJ (2011). The effectiveness and feasibility of videoconferencing technology to provide evidence-based treatment to rural domestic violence and sexual assault populations. Telemedicine and e-Health, 17(4), 309–315. doi: 10.1089/tmj.2010.0147 [DOI] [PubMed] [Google Scholar]
  11. Hoth AB, Shafer C, Dillon DB, Mayer R, Walton G, & Ohl ME (2019). Iowa TelePrEP: A Public-Health-Partnered Telehealth Model for Human Immunodeficiency Virus Preexposure Prophylaxis Delivery in a Rural State. Sexually Transmitted Diseases, 46(8), 507–512. doi: 10.1097/OLQ.0000000000001017 [DOI] [PubMed] [Google Scholar]
  12. Livingood WC, Coughlin S, Bowman W, Bryant T III, & Goldhagen J (2007). Application of economic impact analysis to a local public health agency and its “Academic Health Department”. Public Health Reports, 122(1), 27–36. doi: 10.1177/003335490712200104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. MacLeod KJ, Marcin JP, Boyle C, Miyamoto S, Dimand RJ, & Rogers KK (2009). Using telemedicine to improve the care delivered to sexually abused children in rural, underserved hospitals. Pediatrics, 123(1), 223–228. doi: 10.1542/peds.2007-1921 [DOI] [PubMed] [Google Scholar]
  14. McBain RK, Sousa JL, Rose AJ, Baxi SM, Faherty LJ, Taplin C, Chappel A, & Fischer SH (2019). Impact of project ECHO Models of Medical Tele-Education: A systematic review. Journal of General Internal Medicine, 34(12), 2842–2857. doi: 10.1007/s11606-019-05291-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Miyamoto S, Dharmar M, Boyle C, Yang NH, MacLeod K, Rogers K, Nesbitt T, & Marcin JP (2014). Impact of telemedicine on the quality of forensic sexual abuse examinations in rural communities. Child Abuse & Neglect, 38(9), 1533–1539. doi: 10.1016/j.chiabu.2014.04.015 [DOI] [PubMed] [Google Scholar]
  16. Moy E, Garcia MC, Bastian B, Rossen LM, Ingram DD, Faul M, Massetti GM, Thomas CC, Hong Y, & Yoon PW (2017). Leading causes of death in nonmetropolitan and metropolitan areas—United States, 1999–2014. MMWR Surveillance Summaries, 66(1), 1. doi: 10.15585/mmwr.ss6601a1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Pammer W, Haney M, Wood BM, Brooks RG, Morse K, Hicks P, Handler EG, Rogers H, & Jennett P (2001). Use of telehealth technology to extend child protection team services. Pediatrics, 108(3), 584–590. doi: 10.1542/peds.108.3.584 [DOI] [PubMed] [Google Scholar]
  18. Patterson D, & Tringali B (2015). Understanding how advocates can affect sexual assault victim engagement in the criminal justice process. Journal of Interpersonal Violence, 30(12), 1987–1997. doi: 10.1177/0886260514552273 [DOI] [PubMed] [Google Scholar]
  19. Pennsylvania Office of Rural Health website. Retrieved December 21, 2020 from https://www.porh.psu.edu/hospital-and-health-system-initiatives/critical-access-hospitals/
  20. Potter A, Mueller K, MacKinney C, & Ward M (2014). Effect of tele-emergency services on recruitment and retention of US rural physicians. Rural and Remote Health, 14(3), 2787. doi: 10.22605/RRH2787 [DOI] [PubMed] [Google Scholar]
  21. Price LE, Noulas P, Wen I, & Spray A (2019). A portal to healing: Treating military families and veterans through telehealth. Journal of Clinical Psychology, 75(2), 271–281. doi: 10.1002/jclp.22720 [DOI] [PubMed] [Google Scholar]
  22. SAS Institute Inc (Producer). (2014). SAS software Version [9.4].
  23. Schwamm LH, Audebert HJ, Amarenco P, Chumbler NR, Frankel MR, George MG, Gorelick PB, Horton KB, Kaste M, & Lackland DT (2009). Recommendations for the implementation of telemedicine within stroke systems of care: A policy statement from the American Heart Association. Stroke, 40(7), 2635–2660. doi: 10.1161/STROKEAHA.109.192361 [DOI] [PubMed] [Google Scholar]
  24. The Center for Rural Pennsylvania website. Retrieved December 21, 2020 https://www.rural.palegislature.us/demographics_rural_urban.html
  25. Thiede E, & Miyamoto S (2020). Rural availability of sexual assault nurse examiners (SANEs). Journal of Rural Health. doi: 10.1111/jrh.12544. 2020 Dec 6. Online ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. United States Census Bureau. (2017). What Is Rural America? Retrieved December 21, 2020 https://www.census.gov/library/stories/2017/08/rural-america.html
  27. United States Government Accountability Office. (2016). Sexual Assault: Information on training, funding, and the availability of forensic examiners. Retrieved December 21, 2020 from https://www.gao.gov/assets/700/695914.pdf
  28. Walsh WA, Meunier-Sham J, & Re C (2019). Using Telehealth for sexual assault forensic examinations: A process evaluation of a national pilot project. Journal of Forensic Nursing, 15(3), 152–162. doi: 10.1097/JFN.0000000000000254 [DOI] [PubMed] [Google Scholar]
  29. Weinstein RS, Lopez AM, Joseph BA, Erps KA, Holcomb M, Barker GP, & Krupinski EA (2014). Telemedicine, telehealth, and mobile health applications that work: Opportunities and barriers. The American Journal of Medicine, 127(3), 183–187. doi: 10.1016/j.amjmed.2013.09.032 [DOI] [PubMed] [Google Scholar]
  30. Zhou C, Crawford A, Serhal E, Kurdyak P, & Sockalingam S (2016). The impact of project ECHO on participant and patient outcomes: a systematic review. Academic Medicine, 91(10), 1439–1461. doi: 10.1097/ACM.0000000000001328 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Data_2
Supplemental Data_3
Supplemental Data_1

RESOURCES