Mental Illness Training on the Internet for Nurse Aides: A Replication Study

Abstract

Internet training courses for Nurse Aides (NA) in long term care facilities (LTCs) have been shown to be effective., Little is known, however, about Internet training effects on NAs in a non-research context, or about continued utilization of an available training program. In this research, a replication study was conducted with the Internet training program Caring Skills: Working with Mental Illness. Three LTCs provided the training to all NAs, each within a one-month interval scheduled during consecutive months. Supervisors were interviewed subsequently about their experiences organizing and supervising the training. Participants in all three LTCs showed positive pre-posttests effects with large effect sizes on situational knowledge and self-efficacy, and knowledge about mental illness. Users rated the program highly, and they indicated that it would improve quality of their resident care. Supervisors praised the training, and they said NAs were using recommended training behaviors. Although available to all staff, nursing and other staff made little use of the training in subsequent months. Training for NAs on the internet thus appears feasible, and it is perceived to be beneficial for resident care. Plans for continued utilization and dissemination of best practices to other staff, however, should be integrated when planning for staff training on the Internet.

Introduction

Dementia, depression and anxiety disorders are common in LTCs, occurring more frequently than among community-dwelling older adults (Seitz, Purandare, & Conn, 2010). Among LTC residents, 68% have been reported to have cognitive impairment, and 41% show moderate-severe impairment (DHHS, 2009). Depression alone may affect 17% (Davison, et al., 2007) to 30% (Tabloski, 2006) of LTC residents and less than half of those with major depressive disorder are diagnosed and treated (Davison, et al., 2007). Up to 90% of patients with dementia are thought to have psychiatric co-morbidities (Lyketsos & Olin, 2002; Plassman, et al., 2007), which can exacerbate the risks of problematic behavior (Leonard, Tinetti, Allore, & Drickamer, 2006). Additionally, the census of younger individuals (i.e., 21–65 years of age) have increased in LTCs from 11.1% in 1999 to 13.6% in 2008 (DHHS, 2009), but they are often undiagnosed and without adequate treatment for mental illness (Sedensky, 2011; DHHS, 1999; 2002).

Despite the high incidence of psychiatric disorders in LTCs, direct care staff are inadequately trained in both geriatrics and mental illness (Glaister & Blair, 2008; Institute of Medicine, 2008; Molinari, et al., 2008; Morris & Mentes, 2006). The repercussions for untrained care workers subjected to challenging behaviors (i.e., physical or verbal abuse) may include anger, stress, fear, job dissatisfaction, and decreased feelings of safety (Gates, Fitzwater, & Succop, 2003; Gates, Fitzwater, & Deets, 2003; Gates, Fitzwater, & Meyer, 1999). Negative experiences increase the likelihood that staff will fear or avoid residents with mental illness (Gates, Fitzwater, & Succop, 2003; Gillespie, Gates, Miller, & Howard, 2010; Molinari, et al., 2008; Park, 2010), which can decrease the quality of resident care.

Training of NAs promotes quality of care for residents with mental illness and improves job satisfaction (Glaister & Blair, 2008; Moyle, Hsu, Lieff, & Vernooij-Dassen, 2010; Yeatts, Cready, Swan, & Shen, 2010), and brief communication skills training can positively impact staff-resident interactions (Williams, Kemper, & Hummert, 2003; Williams, 2006; Williams & Herman, 2011). Research is lacking, however, on how to translate successful research into effective training programs in health care settings (Feldstein & Glasgow, 2008). The potential to deliver behaviorally focused training for NAs on the internet has been demonstrated (Irvine, Billow, Gates et al., 2012a, b; Irvine, Billow, Bourgeois et al., 2012; Irvine, Billow, Eberhage, et al., 2012; Irvine, Bourgeois, Billow et al., 2007), with one study showing decreases in resident assaults after training (Irvine, Billow, Gates, et al., 2012a). We know of no research, however, into how a Web training program might be integrated into the work life of a LTC. Also unknown is how an available Web training program might be utilized on an on-going basis by nursing or other staff.

A prototype mental illness training program for NAs on the Internet showed significant effects in a small pilot study (Irvine, Billow, Bourgeois, et al., 2012). Here we report on a replication study to examine the training effects of the finished Internet mental illness training program, when delivered as part a LTCs training regime. The program evaluation goals were to evaluate NA: (a) changes in situational knowledge and self-efficacy to work with residents with mental illness, (b) changes in knowledge about mental illness, and (c) acceptance of the training. Also examined were nurse trainer (NT) experiences implementing the training, and subsequent use of the training by NAs and staff.

Methods

Overview

The training used in this research was designed to develop behavioral skills and confidence to deal with symptoms exhibited by a resident, regardless of the mental illness(s) that might be causing the behavior. The prototype mental illness training (Irvine, Billow, Bourgeois, et al., 2012) was redesigned for this research by dividing it into three courses on fundamentals: Knowledge About Mental Illness, Fostering Caregiver-Resident Relationships, and Behavioral Skills to Work with Mental Illness Behaviors. A total of 8 new courses were also developed. Five courses applied the fundamental skills and knowledge to a specific mental disorder (i.e., depression, anxiety, manipulation, confusion, false beliefs), two dealt with specific behavioral issues (i.e., aggressive resident behaviors; danger to self), and another course promoted caregiver self-care after emotional interactions with residents. Minimum total viewing time for all 11 courses was approximately four hours.

The third author, employed by a large nursing home chain, assisted with preparation of the research proposal, but had minimal input on content development. Research staff required that two fundamental courses be included in the training, and corporate executives selected two additional disorder-specific courses (see below) to be included in the evaluation. The sample of four courses had an estimated viewing time of 1.5 hours, which could extend longer depending on a users interactive learning pace and reading ability. We were allotted a total of two hours, including pre and posttest assessments.

In this research, the training curriculum was evaluated as a “real-world” effectiveness study (Flay & Sobel, 1983), replicated in three LTCs. The same curriculum was previously evaluated by a nationally recruited sample of licensed LTC staff on the Internet (Irvine, Billow, Eberhage, et al., 2012).

Research Design

Three LTCs in different geographical regions of the U.S. (i.e., California, Nebraska, and Pennsylvania) were recruited for the research by the corporate owner. Each LTC was randomly assigned to participate by training all NAs during one of three consecutive months (Figure 1). The Nurse trainer (NT) at each site was advised that a maximum of 2 hours might be needed for the training, and that multiple visits to complete the training was acceptable. The NTs scheduled the NAs for visits without consultation from the research team.

Figure 1.

Research design depicting the NA-training during consecutive months in the participating LTCs.

The Internet Program: Caring Skills: Working with Mental Illness

The training program consisted of four Internet courses presented in a Learning Management System format, which is often used for continuing education courses. With the help of experienced clinicians and behavioral researchers, the courses were designed to be 10–30 minutes in length, and to embrace a person-centered care philosophy (Brooker, 2004, 2007). That is, a resident’s potentially problematic behavior was viewed as an expression of an unmet need, which the resident could not adequately communicate. The training included video modeling, testimonials, narration, and short on-screen text elements written at the 2^nd– 6^th grade reading level. Each course included up to 10 multiple choice response test items with audio and text feedback summarizing the correct answer. Except for log in, keyboarding was not required (i.e., navigation and responses to questions required only mouse clicks).

Two courses (i.e., About Mental Illness; The A.I.D. Care Strategy) were designed to provide fundamental knowledge and communication skills. In About Mental Illness the narrator presented brief descriptions of common mental disorders (i.e., anxiety disorders, schizophrenia and psychotic disorders, mood disorders, cognitive disorders, and personality disorders). Additional content, derived from mental health websites, was designed to dispel common myths about mental illness by presenting statements (e.g., “Residents with mental illness are often violent,” “Depression is normal for residents”). In response to each statement, users clicked on icons labeled Myth, Fact, or Unsure (which was scored as incorrect). An animation with audio and text scored the choice as correct or incorrect and provided information to reinforce the facts or dispel myths.

The A.I.D. Care Strategy course provided video based skills training for interactions with residents displaying behaviors possibly caused by mental illness. The video models demonstrated a response strategy called A.I.D., which has been used in previous training research (Irvine, Billow, et al., 2012; Irvine, Billow, Bourgeois, et al., 2012; Irvine, et al., 2007). In brief, “A” stands for Assess the situation, “I” stands for Investigate to calmly interrupt the behavior with a friendly greeting, “D” stands for Do something, which includes acknowledging the resident’s emotion and finding a suitable redirection.

Additional courses, Care Strategies for Manipulative Behavior and Care Strategies for Aggressive Behavior, modeled the application of the A.I.D. principles to specific resident behaviors (e.g., manipulation: resident attempts different verbally manipulative techniques to obtain snacks restricted by the care plan; aggression: angry resident demands to leave). Content on symptoms and the importance of proactive recognition of symptoms was presented, and use of active listening and validation techniques were modeled. Multiple exemplars of sample responses and redirection ideas were interactively available by clicking an on-screen link.

Procedures

This study was reviewed by an Institutional Review Board and met criteria as acceptable research that presented minimal risk to human subjects. It was then designated as “Exempt from further review” because the training was to be included as part of each LTCs training regimen on company time. That is, participants were paid by their employer do the training. Participants were given a unique username and password by the NT, allowing the research team anonymously to collect program use and pre- and posttest data. Results were made available to participating LTCs and corporate executives only in summarized form.

The first author visited each site to assure the availability of suitable private space for the computer training. Executive Directors (ED) and NT at each site reported viewing and being impressed with the training, and they expressed their resolve that all care staff would take the training. They were encouraged by the first author to have supervisory staff preview the training before the NAs in order to be able to support and answer questions of NAs who saw it. The ED and NT were informed that a prototype (Irvine, Billow, Bourgeois, et al., 2012) had shown positive training effects on both NAs and nursing staff, and that the Web training would remain available for free use by all staff for six months following the initial training.

The research team purchased two computers ($700 each) for each LTC. Earphones were provided for user privacy and to minimize distraction while using the computers. The Internet connection was established by the cable TV provider. Additionally, each LTC was compensated $5,000 for possible expenses related to the training.

All NAs were to be scheduled during the first two weeks of the training-month assigned to the LTC. The scheduling process was left up to the NTs. The NT assigned NAs for the training depending on the current staffing situation in the LTC. To ensure their entire NA staff completed the program within the allotted two-week period, the NTs sent out reminders, and in Nebraska provided incentives of snacks or sodas to encourage the NA to appear for the training. NAs who did not appear at the assigned time were rescheduled. We were informed that there were no consequences for missing an assigned training time. The NAs received their usual pay or overtime if appropriate for their training time, and each received training credit for time spent. All LTCs completed the training within their assigned month, but none did so within the requested two-week period.

Measures

Users saw a menu of six courses. Course #1 and #6 were a pre- and posttest respectively, and each was designed to take about 5 minutes of response time. The assessments were designed to measure changes in knowledge and self-efficacy, which according to social cognitive theory is linked to behavior change (Bandura 1969, 1977). The pretest included nine demographic items, which were replaced by eight user acceptance items at posttest.

All content-related items included a response choice of Decline to answer. This response was chosen a total of seven times for self-efficacy items and five times for knowledge items across pre- and posttests (see below).

In lieu of behavioral observations, measures are needed to assess training effects in the context of a NAs caregiving role. Video situation testing (VST), with response items linked to short video vignettes of problematic resident behavior, have shown positive training effects both with NAs and licensed staff (Irvine, Billow, et al., 2012; Irvine, Billow, Gates, et al., 2012a, b; Irvine, Billow, Bourgeois, et al., 2012; Irvine, Billow, Eberhage, et al., 2012; Irvine, et al., 2007). VSTs were not, however, possible in this research for technical reasons. Instead, three caregiving situations were portrayed in photos, which were adapted from VSTs used in our previous research as cited above. Photos showed: (1) a female resident grabbing a male in a chair with the caption Female resident shouts “Get out of my chair!,” (2) an emotional resident in a wheel chair, rubbing their legs with text Frightened resident cries out, ”Bugs! Bugs! All over me! HELP!,” and (3) A male resident with an upraised cane while female resident cowers nearby in a wheel chair, with text Male resident swings cane, and SCREAMS!! Resident in chair is in danger.

Based on our previous research, and given the limited time available to evaluate training effects, we chose to assess the situational knowledge and self-efficacy of the trainees. We felt that situational knowledge was a measure of the user’s understanding of what to do in a given behavioral context with a resident, and self-efficacy was the best measure of the user’s perceived capability to deal with the given situation. We also added items to assess NA knowledge of mental illness.

Situational self-efficacy

The first two photo-situations included two self-efficacy response items: If you were faced with the situation pictured above, how confident are you that you: (a) would know what to do? and (b) could successfully decrease the resident’s agitation. They were assessed on 5-point scale from 1 = not at all confident to 5 = extremely confident, and a decline to answer option. If decline to answer was endorsed, the item was treated as missing data. A situation self-efficacy scale was created by taking the mean across the four items. The scale showed good internal consistency (alpha = .86).

Situational knowledge

The third photo-situation included three multiple-choice knowledge items, each with 6 choices. The stems were (a) What should you do first? (b) What is the first thing you should say to the resident? and (c) What should you do to stop this agitated behavior?

Knowledge of mental illness

Six additional items adapted from Irvine, Billow, Eberhage et al. (2012) tested knowledge of statements of common myths and facts about mental illness (e.g., Mental illness is a “disease,” just like diabetes and cancer are diseases.). Response options for each statement included Myth, Fact, Not Sure, and decline to answer.

For the nine total knowledge items, the number answered correctly was divided by the number of items answered to derive the proportion of knowledge items correctly answered. Not Sure was considered incorrect and decline to answer was treated as missing data. Internal consistency ratings are not appropriate for proportion of items scored correct.

Program Acceptance

At posttest, participants responded to eight statements on a five-point scale from 1 = strongly disagree to 5 = strongly agree, decline to answer. Five items were about opinions of the training (i.e., I can apply the training to my job; I learned a lot from the training; The video scenes helped me learn; Compared to an inservice on a similar topic, I would learn more using a training website like this; I would recommend this training to others.) The stem for three additional items, which were designed to measure more global reactions to the training, was: Because [company name] made this training available. Items included I can provide better care for residents, I feel more satisfied with my job, and I feel like my company cares about me.

NT Interviews

At the conclusion of the training-month for each LTC, brief follow-up telephone interviews were conducted with each NT to gather qualitative data about their experiences during the training. Ten open-ended interview questions addressed NT time spent scheduling participants, when program use by NAs occurred, incentives or prompts to encourage participation, NAs computer experience, and the NTs opinion of training effectiveness and value. The interviews, which were conducted during working hours, took 15–20 minutes to complete. The interviewer did not attempt to verify the basis for estimates or opinion- statements by the interviewees.

Data Analysis

A paired t-test was used to assess change in pretest to posttest measures. Effect size computations complement inferential statistics (i.e., p-values) by estimating the strength of the relationship of variables in a statistical population. Reporting effect sizes facilitates an understanding of the substantive, as opposed to just the statistical, results of a study and allows for comparison of effects across different studies. A partial point-biserial r was used as the measure of effect size using the following convention: .14 small, .36 medium, and .51 large (Rosnow & Rosenthal, 2008).

Models were run separately for each of the three sites: Pennsylvania (n=47, 34%), California (n=45, 33%), and Nebraska (n=45, 33%), and for the entire sample and. All 133 participants completed the pretest and 120 (90%) completed the posttest. Participants that completed both assessments were compared to those who completed only the pretest on all baseline outcome measures, demographic characteristics, and study site. No significant differences (at p<.05) were found indicating the missing at random assumption remained tenable. Thus, one fully-imputed data set was generated as it produces more accurate estimates than other missing data techniques like listwise deletion or last-observation carried forward (Schafer & Graham, 2002).

Results

The sample was mostly female (89%), between the ages of 21 and 45 years (64%), and minority (56%; Table 1). Fewer than half had completed high school education (46%), and most had an annual family income less than $40,000 (57%). for descriptive characteristics.

Table 1.

Participant Characteristics.

	N	%
Sex
Female	119	89.5
Male	12	9.0
Did not respond	2	1.5
Age
18–20 years	8	6.0
21–35 years	50	37.6
36–45 years	35	26.3
46–55 years	24	18.0
56–65 years	12	9.0
Over 65 years	2	1.5
Did not respond	2	1.5
Hispanic/Latino ethnicity
Yes	48	36.1
No	77	57.9
Did not respond	8	6.0
Race
American Indian or Native American	6	4.5
Asian	9	6.8
Native Hawaiian or Pacific Islander	2	1.5
Black or African American	9	6.8
White or Caucasian	58	43.6
Mixed race	8	6.0
Other	29	21.8
Did not respond	12	9.0
Highest level of education completed
Less than a high school diploma	15	11.3
High school diploma	61	45.9
Some college, associate’s degree, trade school, military	47	35.3
Military	1	0.8
College degree (BA, BS)	4	3.0
Did not respond	5	3.8
Annual household income
Less than $20,000	20	15.0
$20,000–$39,999	61	45.9
$40,000–$59,999	14	10.5
$60,000–$79,999	13	9.8
Did not respond	25	18.8
Amount of experience caring for adults in a professional setting
0–1 month	1	0.8
2–6 months	7	5.3
7–12 months	7	5.3
1–3 years	23	17.3
4–10 years	44	33.1
More than 10 years	47	35.3
Did not respond	4	3.0
Amount of formal training for residents with a mental illness
None	6	4.5
Very little	15	11.3
Some	34	25.6
A moderate amount	44	33.1
A lot	31	23.3
Did not respond	3	3.3
Number hours per week using a computer at home or at work
0 hours	25	18.8
1–4 hours	63	47.4
5–10 hours	20	15.0
11 or more hours	21	15.8
Did not respond	4	3.0

Results of the within-subject paired t-tests (see Table 2) showed significant pretest to posttest increases in both self-efficacy and knowledge. Significant effects were shown within each site as well as for all 133 participants. Associated effect sizes were all large.

Table 2.

Pretest to Posttest Analysis: Self-Efficacy and Knowledge

	Pretest		Posttest		Test Statistics
	Mean	SD	Mean	SD	t-value	p-value	r
California (n=45)
Self-efficacy	3.69	0.93	4.12	0.69	3.60	<.001	.48
Knowledge	0.38	0.16	0.63	0.16	8.86	<.001	.80
Nebraska (n=41)
Self-efficacy	3.61	0.81	4.23	0.69	5.99	<.001	.69
Knowledge	0.48	0.13	0.78	0.12	12.53	<.001	.89
Pennsylvania (n=47)
Self-efficacy	3.69	0.70	4.07	0.67	4.54	<.001	.56
Knowledge	0.55	0.15	0.73	0.16	9.01	<.001	.80
All Sites (N=133)
Self-efficacy	3.67	0.81	4.13	0.68	7.91	<.001	.57
Knowledge	0.47	0.17	0.71	0.15	16.55	<.001	.82

r = point biserial correlation; .14 small effect, .36 medium effect, and .51 large effect (Rosnow & Rosenthal, 2008).

Program Acceptance

Approximately 90% of the participants “agreed” or “strongly agreed” with each program acceptance statement thus indicating a high degree of acceptance (Table 3). Total mean scores by item ranged from 4.07 (SD .89) for My company cares about me to 4.45 (SD .72) for I can apply the training to my job. A total of 95.8% (M 4.45; SD .72) agreed or strongly agreed, that the training made them able to provide better care for residents.

Table 3.

Program Acceptance.

How much do you agree or disagree with the following statements?	Strongly disagree	Disagree	Neither agree or disagree	Agree	Strongly agree
	%	%	%	%	%	Mean	SD
I can apply the training to my job.	1.7	0.8	0.8	44.1	52.5	4.45	0.72
I learned a lot from the training.	1.7	0.0	3.4	45.8	49.2	4.41	0.72
The video scenes in the training helped me learn.	1.7	0.0	2.5	44.9	50.8	4.43	0.71
Compared to an in-service on a similar topic, I would learn more using a training website like this.	1.7	3.4	7.6	44.1	43.2	4.24	0.86
I would recommend this training to others	2.5	0.0	3.4	43.2	50.8	4.40	0.79
Because this training was made available I will be able to provide better care for the residents.	1.7	0.0	2.5	51.7	44.1	4.36	0.70
Because this training was made available I will feel more satisfied with my job.	1.7	0.0	7.6	52.5	38.1	4.25	0.74
Because this training was made available I feel like my company cares about me.	2.6	4.3	7.7	54.7	30.8	4.07	0.89

SD = standard deviation.

Use of Training Website

Program log files were used to describe the percent of participants who completed each course, the amount of time viewing each course (see Table 4), and subsequent log-ins. Across all sites approximately 83% of the participants completed all four courses and the average number amount of time viewing the site was 84.74 minutes (SD = 28.24). A one-way analysis of variance showed total time spent viewing the site statistically differed between the three sites (F2, 129] = 7.48, p<.001). Follow-up pairwise comparisons showed the California site spent significantly (at p<.05) more time using the program compared to the Pennsylvania site. No other comparisons significantly differed.

Table 4.

Percentage of participants completing each course and minutes viewed.

	Completed the Course	Minutes Viewed
Course Description	%	Mean	SD	MIN	MAX
All Sites (N=133)
1. About mental illness	97.8	17.15	6.44	1.38	42.99
2. AID care strategy	92.5	29.53	9.74	2.48	55.32
3. Decreasing risks of aggressive behavior	91.0	22.00	8.70	1.60	41.37
4. Care strategies for manipulative behaviors	85.8	17.61	8.84	1.25	44.96
All four courses	82.8	84.74	28.24	7.01	157.77

SD = standard deviation, MIN = minimum, MAX = maximum.

Alls LTCs were offered continuing free access to the training website for use for at least six months after the training, with logins provided for all employees in each building. After seven months, a total of nine log-ins were recorded, including six in the Pennsylvania, with 5 occurring in the month following the training, two in Nebraska, and one in California.

NT Interviews

The NTs spent one–four hours scheduling NAs for the training, and from 10–15 minutes per user to a total of 2–3 hours helping NAs use the computers to do the training (Table 5). The California LTC scheduled 75% of the program visits during working hours, while the other two LTCs primarily used overtime for the training. Most of the NAs made only one 2-hour visit to the program. Computer inexperience among NAs was as high as 40% in California, where 10% of NAs were non-native English speakers who needed help with terminology. Computer inexperience level was estimated at 2% in Pennsylvania where NAs routinely used the company’s Internet educational system, which was available to all three LTCs. A written user’s guide placed next to each computer appeared to have been well-read at all three sites. The NTs estimated that more than 75% of NAs were using the recommended training skills with residents daily in Nebraska and California. The percentage increased to at least 80% in problem situations. All three NTs responded that it was very important to use the training as part of new employee orientation.

Table 5.

Results from interview with Nurse Trainers after completion of the research.

	California	Nebraska	Pennsylvania
How much time did you spend?
Looking at program	2 hrs	1hr 40 min	4 – 4.5 hrs
Scheduling NAs	4 hrs	Occurred over 2–3 days	1 hr
Prompting/ reminders	Occurred over several days	Occurred over 2 weeks	2 hrs/shift
Assisting computer users	15 min/user	10–15 min/user	2–3 hrs
What % of the program use by NAs for training occurred in the following time periods?
During work	75	5	0
Before the shift	5	50	30
After shift	10	40	55
Special trip (day off)	10	5	15
What techniques did you use to encourage your NAs to use the program?
Incentives to encourage participation	none	Special treats like snacks and soda	none
Prompting/reminders	Yes. Had to remind NAs or they would forget	Lots and Lots	A ton of reminding and hounding;
Different strategies by shift?	No (but was instructed by supervisor to encourage use of overtime)	No	No
Offered one 2-hour session?	Most in one 2-hr visit; some in 2 visits	Most in 2 hr visit; ~30% in 3 visits	Both 1 & 2 hrs
What % of NAs were obviously inexperienced with computers?
	40%	2 % (most used company’s Internet education system)	20%
What % of users needed assistance with using the mouse
	10%;	0	5%
What % of users needed continuing assistance/supervision to use the program? examples?
	10% (non-English speakers; terminology throughout)	0	20%; log in reminder; navigation help
What % used the printed User Guide at the computer
	25%	75% (very worn by the end of training)	75% (was dog eared by the end)
What % of NAs are using the training with the residents as part of daily care?
	75%	80–90% used skills every day	”No idea”
What % of NAs are using the training with the residents –in problem situations?
	90%; Many said they do not tie residents shoes from in front any more	80–90%; resident moods and behavior improved were	”No idea”
If you could make the decision, how important would it be to use the training as part of orientation for new hires in the future? 1–5 scale; 1 = Not important at all; 5 = Very important
	Very important; Hoping to “find the time” to incorporate	Very important; would love to incorporate it	Very important it

Discussion

The positive findings and large effect sizes in this research support other reports on the effectiveness and benefits of training for NAs on the Internet (Breen, et al., 2007; Hobday, Savik, Smith, & Gaugler, 2010; Irvine, Ary, & Bourgeois, 2003; Irvine, Billow, Gates et al., 2012a, b; Irvine, Billow, Bourgeois, et al., 2012; Irvine, Billow, Eberhage, et al., 2012; Irvine, et al., 2007; MacDonald, Stodel, & Casimiro, 2006). The high user acceptance ratings for the mental illness training, especially that it improves quality of resident care, NA job satisfaction, and feeling cared-about by the employer, further suggest the value of the training. These findings are similar to those for licensed LTC staff who saw the same training program (Irvine, Billow, Eberhage, et al., 2012), and for NAs viewing Web training programs on reacting to aggressive resident behaviors (Irvine, Billow, Gates, et al., 2012a, b).

This research presents an early look at issues related implementing a computerized training in a LTC. The logistics of computer purchase and Internet connection were easily handled in this and other research (Irvine et al., in press). An evaluation instrument was embedded in the course-structure, which provided accountability and summary data as reported here. One LTC scheduled 75% of the training visits during working hours, but the others relied on overtime hours, which does not seem economically sustainable. The compressed time frame for this research, targeted at two weeks to train all NAs, may have influenced scheduling choices, and it seems reasonable to assume that a longer training timeline would provide more opportunities to schedule the training without resorting to overtime. A sizable percentage of users self-reported to be inexperienced, with fewer needing assistance to use the computer mouse, and the need for continuing assistance by the NT varied. Techniques recommended in the training were being used during daily care, more so with problematic resident behaviors, which has not been reported previously. The potential value of the training as a new-employee orientation to demonstrate normative skills for the LTC was recognized by the interviewees and has been recognized in the literature (Hobday et al., 2010; Irvine, Billow Gates et al., 2012a). In sum, Web training appears logistically feasible, and NAs will put the training skills to use, but supervisors may need to monitor inexperienced computer users during training.

Despite positive appraisal of the training value by the NTs and voiced support by the LTC administrators, follow-up usage was disappointing. The dearth of logins indicates little use of the training in subsequent months even though it was available for free. Since the nursing staff in LTCs are reported to be under-trained in geriatrics and mental illness (Glaister & Blair, 2008; Institute of Medicine, 2008; Irvine, Billow, Bourgeois, et al., 2012; Irvine, Billow, Eberhage, et al., 2012), a consequence of not seeing the training meant that supervisors were ill-prepared to provide oversight and problem solving support to their NAs who had seen it. Without the training, supervisors and other non-direct care staff also would be less prepared to interact with residents with mental illness. The reason why a free and effective training resource was not used more is difficult to understand.

The research reported here provides evidence of the potential difficulty associated with the translation of research into practice. Innovative practices will not necessarily be adopted without a thoughtful plan for sustainable implementation (Feldstein & Glasgow, 2008). In the research described here, NTs were positive about the training, and NAs not only were reported to be implementing recommended training behaviors, but they reported that the quality of their care for residents improved. But still, the website was little used in the next six months, suggesting minimal training diffusion to other staff, or use of the training for new employee orientation.

The PRISM model for translating relevant research into practice in health care settings suggests that for successful diffusion, an infrastructure must be developed to sustain adoption of best practices and to overcome barriers across personnel levels (Feldstein & Glasgow, 2008). The research questions in this study were around testing an innovative training, but an additional lesson learned was that a successful training implementation with one employee group in a LTC will not necessarily translate into training diffusion. Missing was a management plan with built-in accountability to set priorities for diffusion of the training to other staff who interact with residents with mental illness. The plan also should include a transition from implementation to maintenance activities so the innovations will not be lost (Feldstein & Glasgow, 2008). Management support, starting at the top, is crucial to adoption of best practices (Feldstein & Glasgow, 2008; Gallagher-Ford, Fineout-Overholt, Melnyk, & Stillwell, 2011; Melnyk, Fineout-Overholt, Gallagher-Ford, & Stillwell, 2011). Administrative buy-in also is a cornerstone of the culture change movement to improve residential care, which includes the tenets of person-centered care and adoption of best training practices (Rahman & Schnelle, 2008). Culture change, is beneficial to residents, staff, and the organization, which can realize better occupancy rates and improved staff retention, along with competitive advantages in the market place (Crandall, White, Schuldheis, & Talerico, 2007; Doty, Koren, & Sturla, 2008). The nascent field of nursing informatics (Booth, 2006; McCormick, et al., 20072) seems focused on electronic medical records (Staggers, 2012), but in the future, e-Health seems well suited to increase emphasis on training to institutionalize best practices to benefit both staff and patient care.

While the research presented here has interesting results, it also has several limitations. The training effects were evaluated without benefit of a randomized comparison group or follow-up assessment, and only self-report assessments were used. Behavioral measures of NA improvement and of changes in resident behavior are important indices to validate training effects (Irvine, Billow, Gates, et al., 2012a). In vivo behavioral observation and long term follow-up assessments would provide more compelling evidence of training effectiveness.

Because this was effectiveness research (Flay & Sobel, 1983), several methodological weaknesses were inherent in the process. The scheduling and number of visits to the training varied. The impact of a longer training visit vs. two or three shorter visits with a waiting period in between is unknown, and an argument can be made for the benefits of either approach. Further, the order of courses taken was not controlled, and the ability of the NAs to understand and respond to the on-screen survey questions is unknown. While the NAs were expected to be literate by their employer, the NT in the California facility mentioned that some NAs needed assistance with terminology, which raises questions about their responses to the pre- and post-training surveys.

The interviews with the NTs (Table 5) might also be construed to be a limitation because they were relatively brief, and follow-up questions were not included to establish the basis for opinions or estimates or to clarify answers that were unclear or inconsistent with previous responses. For instance, the number of NAs who were obviously inexperienced with computers varied, and did not coincide with those needing continuing assistance. These issues were somewhat confusing given that NAs in all three LTCs, which was owned by the same company, were expected regularly to access to the company’s Internet education system, but one of the NTs reported that 40% of NAs lacked computer experience. A more in-depth survey by an experienced qualitative researcher would be useful to address these issues.

Implications for Practice

This research demonstrates how a behaviorally focused mental illness training on the Internet can be provided in LTCs. Positive results were replicated in geographically distinct regions, and it was well received by the trainees and their supervisors. Translation of this type or innovative research into practice requires management support with preliminary planning to assure sustainable implementation and later maintenance (Feldstein & Glasgow, 2008). Otherwise, as shown in this research, training diffusion probably will not occur.

This and other research has demonstrated that an Internet training program may be effective across personnel levels, including Licensed staff, NAs, and non-direct care staff (Irvine et al., in press; Irvine, Billow, Bourgeois, et al., 2012; Irvine, Billow, Eberhage, et al., 2012). The benefits to staff, and indirectly to residents, of proven replicable individualized training that can be disseminated across staff levels would seem to recommend the widespread adoption of training programs on the Internet. More research is clearly needed, however, into how best to implement this type of adoption across settings.

Accessible summary.

• Research shows that Internet training for direct care workers in long term care facilities is effective, but little is known about how it works in a non-research setting, and if an available training program would continue to be used.

• Three facilities trained all their direct care workers in a replication study with the same training program.

• Trainees showed very positive effects after the training, showing improved self confidence and knowledge about mental illness. Users rated the program highly, and they indicated that the training would improve the quality of their resident care. Supervisors praised the training, and they said care workers were using recommended training behaviors. Although available to all staff, however, nursing and other staff made little use of the training in subsequent months.

• These results suggest that Internet training is feasible, and it is perceived to be beneficial for resident care by trainees. Administrators, however, when they develop plans for staff training on the Internet, need to plan for continued utilization and dissemination of best practices to other staff.