Development and mixed-methods evaluation of a pain assessment video training program for long-term care staff

Abstract

INTRODUCTION:

Inadequacies in pain assessment and management in long-term care have been well documented. Insufficient pain education and inaccurate beliefs about the nature of pain and aging have been identified as possible contributors. The present study addresses the need for improved, efficient and feasible continuing pain education through the use of an assessment training video.

METHODS:

A total of 148 long-term care staff viewed and evaluated the training video. Knowledge changes and pain beliefs were assessed postvideo and at a four-week follow-up. Beliefs about pain, as well as pain and aging, were also examined using multivariate procedures to determine whether these variables influenced participants’ evaluation of the video. Focus groups were also conducted, and transcripts were analyzed using thematic content analysis.

RESULTS:

Pain assessment knowledge improved postvideo and at the four-week follow-up. Participants positively evaluated the content and quality of the video. Individuals who held stronger beliefs (at baseline) about the organic nature of pain provided more positive evaluations. Barriers to implementation of practices in the video identified by the focus groups (and qualitative analysis) included time, workload and resistance to change. Facilitators to implementation included continued management support and observing the benefits to implementation.

DISCUSSION:

The present study provides support for the use of video training. However, based on the focus group results, top-down implementation approaches with ongoing management involvement throughout the implementation process may be needed to achieve sustained changes in pain assessment practices. A model useful for sustained implementation was proposed and discussed, and is hoped to facilitate future research.

As many as 50% of seniors living in the community and 80% of seniors living in long-term care (LTC) facilities experience persistent pain (1). Health professionals agree that the undertreatment of pain in older adults is a growing ethical concern (2–4). The high prevalence of dementia in LTC facilities increases challenges to pain assessment because individuals may not effectively communicate their pain to staff. Pain is less effectively treated in individuals with cognitive impairments (5–8), although systematic examination of pain expressions can facilitate its evaluation (9). Moreover, pain assessment practice guidelines have been developed to improve quality of care in LTC (10–13) but staff unfamiliarity with pain assessment procedures remains a major barrier to effective pain management (14).

Researchers have found that nurses receive inadequate pain education (14–16). Pain-related knowledge among nursing students during or at completion of their training has been deemed to be insufficient (15,17,18). While pain curricula significantly improve pain knowledge among students in health-related programs (19,20), curriculum reviews of Canadian health care programs indicate that pain is rarely covered comprehensively during training (21).

Continuing education (CE) presents an avenue for health care professionals to gain knowledge about effective pain practices (22). Pain-focused CE programs implemented among nurses have led to positive outcomes, including increased pain-related knowledge, improved pain assessment ability and increased documentation of patient pain (18,22,23). CE programs have also been found to increase satisfaction and reduce pain among patients (24,25).

Despite these positive outcomes, CE for LTC staff is limited due to financial and resource barriers (26,27), and rural and remote staff are often isolated from locations offering programs (28). Moreover, LTC staff who attend CE have reported feeling overwhelmed by the amount of information presented and experience uncertainty about how to implement changes on returning to their facility (22,29). A video training approach would address many of these issues by providing the opportunity for staff to review the video as they see fit (eg, more than once) and to watch it at their convenience.

Beliefs about the nature of pain and pain in old age have also been identified as factors influencing pain assessment and management in LTC (25,30–33). LTC nursing directors recognize negative staff attitudes and misconceptions about pain in older adults as major barriers to proper pain management (33). If staff are unaware of their inaccurate beliefs, they may not seek out pain assessment programs, thereby jeopardizing these programs’ utility and efficacy.

Purpose

The primary purpose of the present study was to develop and evaluate a training video on pain assessment in older adults. As such, this format addresses many previously identified barriers to CE (22,26–29). As a corollary, we also aimed to examine the way in which beliefs about the nature of pain influence staff views about the video.

It was expected that video ratings would be positive. In addition, it was hypothesized that nurses would rate the video content more positively than care aides (because of a better preparatory background knowledge base), that participants’ knowledge of pain assessment would increase, that these increases would be maintained over four weeks, and that beliefs about pain would be predictive of the valence of video evaluations.

METHODS

Participants

Care staff (n=148) working in LTC facilities affiliated with two large rural Canadian health regions were recruited to participate in the present study. Demographic characteristics of the sample are summarized in Table 1. Informed consent was obtained and ethics review was completed.

TABLE 1.

Demographic characteristics of the full sample (n=148)

Characteristic
Age, years, mean ± SD	45.34±10.62
Sex
Male	11 (7.4)
Female	137 (92.6)
Job level
Care aide	81 (54.7)
Licensed practical nurse	24 (16.2)
Registered nurse	30 (20.3)
Registered psychiatric nurse	13 (8.8)
Years in current job, mean ± SD	18.41±11.60
Years working in LTC, mean ± SD	14.24±9.56

Data presented as n (%) unless otherwise indicated. LTC Long-term care

Materials and measures

Pain assessment training video:

A training video on pain assessment in LTC with a special focus on individuals with dementia was developed for the present study. The video included a combination of video clips from training sessions on pain assessment in LTC conducted by an expert in the area and on-screen text paired with an audio discussion of the content. Role-played illustrations of pain assessment with an LTC patient, as well as material from interviews with health care staff, were included. The 45 min video discussed the benefits of using pain assessment in LTC, self-report measures of pain assessment appropriate for individuals with mild to moderate dementia, and clinically useful standardized observational approaches (involving use of the Pain Assessment Checklist for Seniors with Limited Ability to Communicate) suitable for individuals with severe limitations in their ability to communicate (34,35). The video concluded with guidelines for assessing pain in seniors with cognitive impairments (10) and suggestions of feasible implementation strategies. A copy of the video can be obtained from the authors.

Demographic information:

Information regarding participants’ sex, age, degree or certification as a nurse, years of experience as a nurse/care aide and years of experience working in LTC was collected. Participants also self-reported (using 0 to 6 Likert scales) the extent of formal training in pain assessment and in the assessment of pain in dementia.

Pain Beliefs Questionnaire:

Beliefs about pain were assessed using a modified Pain Beliefs Questionnaire (PBQ) (36). The original version of this questionnaire consists of two subscales, the Organic subscale (PBQ-O), which addresses beliefs about the nature of pain, and the Psychological subscale (PBQ-P), which evaluates beliefs concerning the relationship between psychological factors and pain. The measure has been modified to include an Aging subscale (PBQ-A), which evaluates beliefs about the relationship between aging and pain (37). Stronger endorsement of psychological pain beliefs and weaker endorsement of organic and aging pain beliefs are considered to reflect a more adaptive understanding of the pain experience. The PBQ-O, PBQ-P and PBQ-A subscales of the PBQ have demonstrated acceptable reliability, with Cronbach’s alphas of 0.73, 0.70 and 0.73, respectively (36,37). In the current sample, Cronbach’s alpha was 0.51 for the PBQ-O subscale, 0.78 for the PBQ-P subscale and 0.69 for the PBQ-A subscale.

Knowledge test:

To evaluate knowledge gains, a 14-item questionnaire was developed that queried video-specific knowledge about pain assessment using a multiple choice format. The knowledge test (KT) was administered prevideo, postvideo and four weeks following training.

Video evaluation questionnaire:

The video evaluation questionnaire (VEQ) (31 items) was developed to evaluate participants’ appreciation of the video and was administered immediately after participants watched the video. The VEQ includes two scales, the Content Quality Scale (CQS; 11 items), which poses questions (0 to 4 Likert format) related to value seen in the video and willingness to implement the practices, and the Technical Quality Scale (TQS; 14 items) which addresses the technical nature of the video and quality of video production (eg, sound quality). Higher scores were indicative of more satisfaction. The CQS includes two subscales. The first examines perceived general value of the information (General Content Quality; G-CQS) and the second examines value seen in the information in terms of the LTC facility and the participants’ specific job (Specific Content Quality; S-CQS). Cronbach’s alphas for the G-CQS, the S-CQS and the TQS were 0.87, 0.89 and 0.89, respectively. The VEQ included two additional questions that queried participants’ willingness to implement the practices described in the video and whether they felt confident in using these practices.

Follow-up questionnaire:

The follow-up questionnaire (FQ) was distributed four weeks following training. This 13-item, 0 to 4 Likert scale questionnaire, developed for the present study, inquired as to whether changes in pain assessment practices had occurred. Cronbach’s alpha for the FQ was 0.91.

Study design

Administrators from participating facilities informed qualifying staff members of the study. Interested participants registered to attend a training session. A total of 13 training sessions were organized across the two health regions. At each training session, participants were asked to complete a demographic information sheet, the KT and the PBQ. Next, the 45 min training video was presented. On its conclusion, participants were asked to complete the VEQ and to repeat the KT. Four weeks following the training, participants were mailed packages containing FQs and KTs. The questionnaires were returned to the researchers using prestamped envelopes.

Focus groups and individual interviews were arranged with a subset of participants who were nominated by the facility administrators as influential leaders within their peer group, as per qualitative research standards (38). In total, six individual interviews (five nurses and one care aide) and four focus groups (10 nurses and 10 care aides) were conducted, audiotaped and transcribed verbatim. Separate focus groups were held for nurses and for care aides. The interview/focus group questions were standardized according to a moderator guide developed for the present study. Questions regarded perceived utility of the video and current pain assessment practices in the facility.

RESULTS

Quantitative results

Video evaluation and group differences:

Mean scores and SDs (Table 2) on the CQS (G-CQS and S-CQS), and TQS of the VEQ were used to determine participants’ evaluation of the video content, the presentation of the information and the technical aspects of the video. Scores reflected satisfaction with the content and quality. Independent-samples t tests were conducted to examine differences between nurse and care aide CQS total score (ie, sum of G-CQS and S-CQS) and CQS subscale scores. Significant differences were found on total CQS score (t[145]=2.75; P=0.007) and on S-CQS (t[145]=3.38; P=0.001), with nurses providing more positive evaluations. A large proportion of participants (85.3%) reported that they would feel confident using the practices taught in the video, and the majority of participants (90.0%) indicated being willing to implement the practices.

TABLE 2.

Descriptive results of the Video Evaluation Questionnaire (VEQ) scales and subscales

VEQ subscale	Nurses (n=67)	Care aides (n=81)
CQS*	33.53±5.31	32.45±5.07
G-CQS	11.98±2.43	11.54±2.21
S-CQS*	22.81±3.32	20.93±3.41
TQS†	41.02±5.77	38.72±5.80

Data presented as mean ± SD. Content Quality Scale (CQS) maximum score = 44; General subscale of the CQS (G-CQS) maximum score = 16; Specific subscale of the CQS (S-CQS) maximum score = 28; Technical Quality Scale (TQS) maximum score = 56.

^*P<0.01;

^†P<0.05

Overall learning:

A two-way mixed factorial ANOVA was conducted to compare scores on the prevideo, postvideo and follow-up KTs, with nurses’ and care aides’ results compared between groups. The main effect of time was significant (F[1.83, 86]=102.81; P<0.001, partial η²=0.54). Pair-wise analyses indicated that scores significantly increased from prevideo to postvideo (P<0.001) and from prevideo to follow-up (P<0.001), but significantly decreased between post-test and follow-up (P<0.001). There was no main effect of job type (ie, nurse or care aide) and the interaction between job level and time was not significant.

Role of beliefs in video evaluation:

To determine whether maladapt-ive beliefs about the nature of pain and pain in old age would be significant contributors to overall video evaluation, multiple regression was used. Three regressions were conducted, a separate one with each of G-CQS, S-CQS and FQ as the dependent variable. The predictors of age, sex, job type (nurse or care aide), years of experience in LTC, extent of past pain assessment training, extent of past pain assessment in dementia training and the three PBQ scores (one score for each subscale) were included. If the overall models were found to be significant, each variable’s unique contribution to the regression was examined using a conservative regression approach in which each predictor’s ability to account for unique variance was examined after all other predictors had been entered into the model. The full model for G-CQS was not significant (F[9, 121]=1.76; P=0.083, R² =0.12). The full model for S-CQS was significant (F[9, 121]=2.15; P=0.03, R²=0.14). The regression results and standardized regression coefficients (β) for this model are summarized in Table 3. An examination of the predictors suggested that job level (nurse or care aide) and PBQ-O accounted for a significant and unique portion of the variance. The full model for FQ was significant (F[9, 68]=2.21; P=0.032, R²=0.23); however, no predictors made an independent and unique contribution.

TABLE 3.

Regression analysis examining the variance accounted for by the predictors for the Specific Content Quality Scale of the Video Evaluation Questionnaire (S-CQS) regression model

Predictors	β	F(10, 21)	P	R2 change
Age	0.06	0.25	0.62	0.00
Sex	−0.03	0.14	0.71	0.00
Job level	0.24	4.93	0.03	0.04
Years of experience	0.03	0.07	0.80	0.00
Extent of past pain training	0.00	0.00	0.98	0.00
Extent of past pain in dementia training	0.05	0.16	0.69	0.00
PBQ organic subscale	0.19	4.08	0.05	0.03
PBQ psychological subscale	0.06	0.43	0.513	0.00
PBQ aging subscale	−0.10	1.06	0.31	0.01

PBQ Pain Beliefs Questionnaire; PDQ Personhood in Dementia Questionnaire

Qualitative results

Following from the results of the self-report measures, qualitative analysis focused on understanding whether the training led to a change in practice and on identification of facilitators and barriers to change implementation; herein, participants’ perceptions of the video’s usefulness are also summarized. Thematic content analysis (38,39) was used to organize the text, first based on whether it referenced video evaluation and utility, or pain assessment in the facility, and subsequently into thematic categories using a bottom-up procedures. To ensure consistency of the coding and trustworthiness of themes, a second coder analyzed a random 15% of transcripts into the established themes. Disagreements were discussed and resolved by consensus. Problematic themes were reconsidered and reorganized. Member checking, implemented to ensure accurate interpretation of the data, was conducted with all focus-group participants (40). Participants stated that the qualitative results were consistent with their experiences.

The discussion regarding video evaluation and utility was consistent with the quantitative findings. Participants were pleased with the training and found the information useful and relevant. However, participants were unable to recall specific details of the practices and did not link their descriptions with behavioural examples indicative of changes in practices (see Discussion section for a recommendation concerning this issue). Participant discussions of pain assessment practices currently in use within the facilities similarly denoted lack of implementation.

The themes related to pain assessment in the facility were examined to provide a better understanding of factors that interfere with or facilitate a change in practices (Table 4). Factors identified as barriers to implementation included insufficient time, insufficient staff, and coworker negativity and resistance to change. Staff who believed that there was sufficient time still identified time as an issue, describing it as challenging to overcome the negativity of staff who perceived time as lacking.

TABLE 4.

Facilitators and barriers to changing practices and sample responses

Facilitators to change
Seeing the benefits to implementation	“Show us how to use it, the benefits. Like what are we going to get out of this that we aren’t doing right now.” (N)
Management support and follow-up	“It seems like all the managers are done at 3:30 [p.m.] and they’re gone, and you can’t carry these things out as a nurse on your own. You’re responsible for so much. When you bring in change you got to over-see it, seven days a week. Not Monday to Friday.” (N)
Barriers to change
Insufficient time	“Certainly I’d like to do it, because it probably would benefit my job in the end, or my coworkers on shift. I’d love to do it, but it comes down to time.” (CA)
Insufficient staff	CA1: Like it’s useful but . . . we don’t always have enough staff
	CA2: And that’s what it’s all coming down to.
	CA3: We don’t have enough staff.
	CA2: And that’s everywhere
	CA1: Bottom line, exactly.
	CA2: Everywhere, for every department. (CA)
Coworkers’ resistance to change and negativity	“I’m thinking of certain people here, and I think, short of an atomic bomb, nothing will change their outlook on life . . . there are some, they would rather die than change. ‘We’ve done it this way, and we’re always going to do it this way, and don’t tell me that I have to change.’”(N)

CA Care aide quotation; N Nurse quotation

Management support was identified as an essential factor to successful changes in practices. Participants indicated that large-scale change had to be initiated by management and that support must be ongoing throughout all phases of implementation. Participants emphasized the necessity of management awareness of the experiences of front-line staff and availability to address concerns. Without sufficient support, participants suggested that the barriers could not be overcome.

Witnessing the benefits to implementation was identified as a second essential factor to successful change. Participants described being open to implementing new approaches if they could see how the changes would improve on existing procedures and address existing pain assessment challenges. Existing challenges – a great source of frustration to staff – included unreliability of patient self-report, incomplete and inconsistent charting, perceived lack of respect for care aides’ reports, family demands and lack of understanding, and physician interference with pain management.

Proposed model of successful change in pain assessment practices

Despite positive attitudes about the training, participants were not implementing the recommended practices. Quinn (41) defines a successful CE program as one that results in a change in practices. Thus, implementation of the practices would represent optimal success of this training program. To improve the success of future training using this program and to increase changes in practices following training, a model, subject to cross-validation by future research and offering a conceptualization of the qualitative findings, is proposed (Figure 1). This model focuses on the two main aspects identified by staff as facilitators to change: continuous management support and the importance of seeing the benefits.

Figure 1).

Proposed model of successful change in pain assessment practices. The dark arrow along the right of the model depicts management involvement in the staff members’ experience. For front-line staff, the process begins with the completion of the training program (Box 1). Participants perceived several benefits to the training program (Box 2). Pre-existing challenges to pain assessment (Box 3) may interfere, making it challenging for staff members to adjust their practices independently. Management should oversee the implementation of practices (taking a consultative team approach) while providing ongoing support to enable actual implementation to occur (Box 4). Barriers to implementation (Box 5) must be addressed. The true outcomes (Box 6) of using the pain assessment practices will occur. Sufficient management support should lead to the positive outcomes (Box 7); however, should negative outcomes be experienced (Box 8), these must be immediately addressed with management support. With the experience of benefits and continued management support, sustained changes should occur within the facility (Box 9)

A perception of how changing practices would alleviate existing challenges was identified as necessary to change. Quantitative results suggest that staff members perceive these benefits immediately following training. The perception of benefits alone did not lead to any lasting change in practices. It is possible that participants initially saw ways in which pain could be better assessed and managed despite existing challenges, but on return to the facility, perceived these issues as too numerous and too overwhelming to enable change. Management support becomes essential to allow staff members to experience the true benefits of the practices through mandated implementation, continuous emphasis of the benefits and recognition of the pain assessment challenges staff members face. To counteract the barriers to implementing new practices, managers must be aware of how implementation is occurring at the front-line level and respond to staff concerns. Managers will influence the degree to which staff members are able to overcome barriers and experience the benefits of the pain assessment practices. While additional research is needed to investigate the proposed model, some of its basic tenets are consistent with other implementation approaches discussed in the literature. For example, Gnaedinger (42) underscored the importance of leadership, a team approach to overcoming obstacles and ongoing training. Moreover, Stolee et al (26) obtained preliminary research support for several aspects of our proposed model including the importance of securing management commitment and on-the-job support.

DISCUSSION

The present investigation provides support for the use of this video-based program as a method of knowledge translation for LTC staff. Video evaluations suggested that, in general, participants were pleased with both the technical presentation of the video and the content and information in the video. Nurses provided more positive ratings, which may be due to differences in training. Care aides receive less training on pain assessment in their initial training program and may be less aware than nurses of the importance of conducting pain assessments.

Knowledge of pain assessment in LTC was significantly greater following the video; however, knowledge decreased between the post-video phase and the four-week follow-up. These findings suggest that this tool can effectively educate staff in evidence-based pain assessment practices, but support the necessity of refresher or booster training, especially in the absence of immediate clinical implementation.

It was expected that beliefs would influence participants’ evaluation of the video and implementation of practices. Organic beliefs resulted in more positive evaluations of the degree to which the content of the video was relevant to participants’ specific job and duties in the LTC setting. It is possible that individuals who more strongly believe in the organic nature of pain found the video most beneficial because it contained information about the role of nonorganic factors in the pain experience.

The content of the video is consistent with evidence-based recommendations and may assist LTC facilities in meeting clinical and policy pain management recommendations (43). The video format addresses many of the previously identified challenges to CE programs (22,27,28) and, with proper implementation, this training tool has the potential to facilitate pain management decision making (44) and to address the ethical concern of the undertreatment of pain in older adults (2).

Our results indicate that many obstacles are present for LTC staff wishing to change pain assessment practices following training. These obstacles are formidable for front-line staff to overcome on their own. Participants in the present study stressed the need for strong, ongoing management support for a training program to result in a change in practices. This idea is further reinforced by the qualitative finding that staff were unable to recall certain details of the practices in the video and suggests that, with central management support, additional training time can be allocated to enable review and consolidation of the knowledge that was conveyed.

Nurse perceptions underscore the need for training programs to account for pre-existing challenges in LTC and provide strategies for mitigation. Bottom-up approaches to CE in which staff are educated in the hope that practices will be implemented have demonstrated moderate success (22,25). The present study suggests that these approaches may not be realistic given the resources and hierarchy of the LTC setting. Bottom-up approaches require a small subset of staff members to advocate for the implementation of practices, which may not be possible given staffing and time limitations. Instead, the findings of the present study support an approach involving consistent and continuous management involvement. With this said, we acknowledge, as a limitation to our conclusions, that data on the perspectives of managers regarding this issue were not available. As such, we encourage future research on the validity of our recommendation concerning the importance of management support. Nonetheless, we do believe that such support, sustained over the long term, would likely result in better pain assessment and management within the facility. An added benefit would be that improved pain assessment and management has been shown to improve workplace environment for the front-line staff and quality of life for LTC residents (45).

LESSONS FOR PRACTICE

Video-based pain assessment training can be an effective way to increase knowledge of pain assessment practices among LTC nurses and care aides. Copies of the video used in the present study can be obtained from the authors. Focus group results suggested that strong management involvement is essential to the implementation of changes in pain assessment practices in a LTC setting.