Evaluation of a Dementia Training Course for Staff of a Center of Dementia Care

Abstract

Introduction

Literature shows poor dementia training and competencies among health and social professionals. Due to the growing prevalence of people with dementia and all the related care demands, specialized training is increasingly needed but must be effective in terms of impact on knowledge, behaviors, and attitudes. We aimed to analyze the impact of a first-level dementia training course for staff of a new specialized center for people with dementia, considering the first three levels of Kirkpatrick’s evaluation framework, namely, staff reaction (satisfaction), skills and learning (knowledge and dementia attitudes), and behavior changes.

Methods

This is a single-center group pre-post design study of a 12-session online course. An online questionnaire was administered to measure satisfaction, expectations, knowledge/learning, attitudes (Dementia Attitude Scale), and new behaviors/practices. We compared perceived knowledge (Wilcoxon signed-rank test) and attitudes (paired t test). Thematic analysis explored new behaviors/practices.

Results

Eighty-five professionals and 1 volunteer were included (median age 31, 92% female). Satisfaction with the training was high (median 4/5). Perceived knowledge improved (median 3–4; p < 0.001). The knowledge test median score was 70.8%. After training, participants showed better attitudes toward dementia (mean 116.5, SD 10.3, to mean 122.2, SD 11.5; p < 0.001). Most (93%) said their behavior/practice changed. Thematic analysis yielded four new behavior/practice dimensions: care provision/interaction, communication, family/caregivers, and self-confidence.

Conclusions

The course improved all dimensions evaluated, suggesting it effectively provides first-level dementia training. This may be transferable to similar settings.

Introduction

The increased burden of dementia is one of the most complex challenges posed by population aging. There are approximately 50 million people living with dementia worldwide and this is set to increase to 82 million by 2030 and 152 million by 2050 [1]. Both the World Health Organization (WHO) and Alzheimer’s Disease International [1] recognize dementia as a global public health challenge and called for governments to define national dementia policies.

International and national organizations have recommended the increase of specialized dementia training to improve care and reduce or prevent burnout among professionals [2]. National dementia policies also acknowledge the need, and this is the case in Portugal [3]. The creation of specialized centers, health or social facilities arises as a response to the high number of people with dementia (PwD), and these must offer services designed to meet the specific needs of PwD. High-quality dementia care requires professionals with good knowledge and skills about PwD and their needs. However, there are a limited number of health and social professionals trained in dementia and a substantial part of the workforce that provide dementia care remains poorly prepared [2]. One operational strategy that can be used is to offer pre-service interprofessional training to guarantee that professionals who work with PwD and their caregivers have the knowledge they need [2].

The literature describes some dementia training courses for health and social staff who work with PwD in a hospital setting [4–9], but none was promoted before the start of the service. Also, the care provided by a hospital is different from a dementia specialized center which aims to promote holistic and integrated care specific to dementia.

The previous studies that analyzed the impact of training, mentioned above, revealed improvements in knowledge [4–6, 9, 10], attitudes [7, 8, 10], self-efficacy, and confidence [4, 6, 7, 9, 10] while reducing the stress associated with caring [11]. However, most evaluations only assess the impact on knowledge and staff satisfaction, with few reporting the impact on attitudes and practical skills.

The Kirkpatrick’s evaluation framework [12] is a comprehensive conceptual model of what works in training and has been applied in dementia [13]. The model includes four levels of evaluation: level 1 comprises the staff reaction to and satisfaction with the training program; level 2 assesses the extent of learning and includes knowledge, skills, confidence, and attitudes; level 3 explores the extent to which staff behavior or practices have changed with training completion; level 4 consists of the impact on outcomes for PwD or their family (e.g., quality of care).

The aim of this study is to analyze the impact of a first-level pre-service dementia training course (a training course that gathers basic and essential contents aimed at professionals with low prior knowledge about the subject) created for staff of a new specialized center for PwD, considering the first three levels of Kirkpatrick’s evaluation framework, namely, staff reaction to and satisfaction with the training (level 1), skills and learning (level 2), and behavior changes (level 3).

Materials and Methods

Study Design

This study used a single-center group pre-post design. An online questionnaire was administered prior to the course (baseline) and up to 1 week after its end (post-training evaluation). The online questionnaire link was sent by email to each professional registered in the training course. All participants were asked to answer the questionnaire individually and a deadline to complete it was defined. Professionals without an e-mail address or with low digital literacy could complete a paper-based questionnaire. In this case, participants could pick up and deliver the questionnaire at a defined location in the center.

Setting and Participants

The study was conducted in a new center for PwD located in Portugal (Europe). This center was built to provide a set of services from diagnosis, rehabilitation to palliative care, grounded in a multidisciplinary and person-centered approach. At the time the course took place, the center was in its final construction stage. The preparation and organization of the center services included the development of training initiatives to increase knowledge about dementia among its professionals. In this context, a mandatory interprofessional dementia course was developed and offered to all the professionals who would work at the center. The study was explained to all professionals by e-mail before the training course. The refusal to participate in the study did not influence course attendance.

Training Course

The course program was created by the direction and coordination boards (doctors, nurses, researcher, occupational therapist) of the center considering the international and national recommendations [1–3], the objectives of the center, and their own knowledge about dementia. The course was conducted online due to COVID-19 pandemic circumstances in April 2021. It lasted 9 h, organized in 12 sessions with 4 sessions per day with 45 min each (Table 1). The sessions were delivered by 12 external dementia experts with backgrounds in nursing, social work, medicine, psychology, occupational therapy, law, health research, and management. The main objective was to offer a first-level pre-service course to give basic knowledge about dementia care to all professionals. On completion of the course, the participants should be able to recognize dementia as a public health priority; identify and distinguish the most common types of dementia, including the main symptoms; demonstrate knowledge about strategies for identification of the PwD needs and interventions for their successful management; understand the consequences in the caregivers and family of PwD; recognize the importance of the teamwork to manage the multidisciplinary needs of PwD.

Table 1.

Description of the sessions and themes covered in the training course

Day	Session/theme
1	Session 1 – dementia care needs: importance and national strategy
	Session 2 – epidemiology and natural course of dementia
	Session 3 – person-centered approach to dementia care
	Session 4 – assessment and management of psychosocial needs
2	Session 5 – assessment and management of physical needs
	Session 6 – communicating with patients and families
	Session 7 – support for family and informal caregivers
	Session 8 – palliative care
3	Session 9 – environment-friendly institutions
	Session 10 – legal and ethical issues
	Session 11 – team work and professionals’ well-being
	Session 12 – research, training, and quality standards

The training was offered through lectures and teaching discussion. Participants could ask questions at the end of each lecture using the online chat function.

Measures

The questionnaire was created specifically for this study and was organized in four sections.

• 1.Sociodemographic information, professional experience, and training in dementia: participants’ characteristics included age, sex, education years, professional category, years of professional experience, professional position, experience caring for PwD (none/professional/personal), and training in dementia (hours).
• 2.Staffs’ reaction and satisfaction – level 1 of Kirkpatrick’s framework [12, 13]: after the completion of training, participants’ satisfaction was evaluated by 7 closed questions using a 5-point Likert scale (1 dissatisfied to 5 completely satisfied): clarity of objectives, achievement of objectives, usefulness of themes, level of knowledge acquired to improve professional performance, course duration, quality and adequacy of the software used (zoom), and the overall course appreciation. Participants were also asked about their expectations, measured by a single question “did the training course meet your expectations?” on a 4-point Likert scale (1 = did not meet the expectations; 2 = partially met the expectations; 3 = completely met the expectations; 4 = exceeded the expectations).
• 3.Skills and learning – level 2 of Kirkpatrick’s framework [12, 13]: this section considered (1) the self-perceived knowledge about the themes, measured at baseline and post-training evaluation by a 5-point Likert scale question (1 none knowledge to 5 full knowledge); (2) the knowledge/learning directly obtained from the course, which was measured by 24 multiple-choice questions after training (2 questions from each theme of the training course) (questionnaire available upon request); and (3) professionals’ attitudes toward PwD assessed by the Portuguese version of Dementia Attitude Scale (DAS) [14, 15] at baseline and post-training. The DAS contains 20 items with a 7-point Likert scale (1 = strongly disagree to 7 = strongly agree). The total score ranges from 20 to 140 with higher scores indicating more positive attitudes. The Portuguese version revealed high internal consistency (Cronbach’s alpha = 0.952 and 0.890) and excellent test-retest reliability (ICC 2.1 = 90.8% and 80.2%) [16]. Originally, the term to address people with cognitive impairment in the questionnaire was people with “Alzheimer’s disease and related dementias” [14, 15]. We found this term difficult to understand and therefore used the Portuguese term for PwD [i.e., pessoas com demência].
• 4.Behavior changes – level 3 of Kirkpatrick’s framework [12, 13]: behavior and practices changes were measured by two close-ended questions asking if the participants considered that their behavior or practice changed immediately after the course (yes/no). Those who answered “yes” were asked to give examples of behavior or practices changes.

Data Analysis

For descriptive statistics, mean and standard deviation were calculated for continuous and normally distributed variables, median and interquartile range for continuous but not normally distributed variables and ordinal variables. Absolute and relative frequencies were calculated for categorical variables. Wilcoxon signed-rank test was used to compare the self-perceived knowledge before and after training, and the paired-sample t test was used to compare the attitudes toward PwD at baseline and post-training. All statistical analyses were performed using IBM SPSS Statistics 27.0, with 0.05 as significance level. Thematic analysis using the six steps approach of Braun and Clarke [16] was used to establish themes when participants shared comments about behavior or practices changes. In the first step, the first author familiarized themselves with the data by reading and rereading the answers. The second phase involved the initial production of codes. These codes were then shortened and organized into potential themes. In the fourth step, researchers reviewed and refined the themes through reading the extracts for each theme. The first author conducted the analysis, and the second and third authors reviewed the findings. Each theme is described with illustrative quotes.

Results

Sociodemographic Information and Professional Experience

One hundred sixteen professionals and volunteers were registered for the training, of whom 101 completed the baseline questionnaire (response rate = 87.1%). After the completion of training, the number of returned questionnaires was 86 (response rate = 85.1%).

Table 2 shows the sample characteristics. Median age was 31 years (IQR 16) and 93 (92.1%) were women. Most participants had high level of education (53.5% with more than 12 years of education). The course enrolled people from operational and executive positions (89.1% and 10.9%, respectively). Healthcare assistants (50.5%) and nurses (33.7%) were the most represented. Most participants had relatively few years of work experience, 72% between 0 and 5 years of work. In regard to previous experience caring for PwD, 40.6% had no professional or personal experience. Most participants had never attended training courses on dementia (64.4%).

Table 2.

Sample characteristics (n = 101)

	n (%)
Age (median [IQR])	31 (16)
Sex
Female	93 (92.1)
Male	8 (7.9)
Years of education
≤9 years	5 (5.0)
10–12 years	42 (41.6)
≥13 years	54 (53.5)
Professional position
Executive	11 (10.9)
Operational	90 (89.1)
Professional category
Healthcare assistant	51 (50.5)
Nurse	34 (33.7)
Doctor	2 (2.0)
Manager	1 (1.0)
Occupational therapist	2 (2.0)
Physiotherapist	3 (3.0)
Volunteer	1 (1.0)
Lawyer	1 (1.0)
Administrative	1 (1.0)
Psychologist	1 (1.0)
Social worker	1 (1.0)
Gerontologist	1 (1.0)
Nutritionist	2 (2.0)
Professional experience
0–5 years	73 (72.3)
6–10 years	14 (13.9)
11–15 years	11 (10.9)
16–20 years	2 (2.0)
≥21 years	1 (1.0)
Experience with PwD (yes/no)
Without experience	41 (40.6)
Professional experience	36 (35.6)
Personal experience	14 (13.9)
Both, personal and professional experience	10 (9.9)
Training in dementia care
0 h	65 (64.4)
1–7 h	8 (7.9)
8–17 h	10 (9.9)
18–89 h	9 (8.9)
90–279 h	3 (3.0)
280+ h	6 (5.9)

Level 1: Staff Reaction and Satisfaction

Globally, the training course was positively evaluated (median 4 very satisfied; IQR 1). Thirty-four participants (39.5%) indicated they were completely satisfied, 44 (51.2%) were very satisfied, and 8 (9.3%) were satisfied. None of the participants reported to be “dissatisfied.” Figure 1 shows satisfaction levels with the objectives, utility in practice, and logistics aspects. Satisfaction with the length of the course was the lowest but still high (median 4, IQR 2).

Fig. 1.

Satisfaction with the objectives, utility in practice, and logistics aspects of the course.

Most participants considered that the course completely met their initial expectations (64.0%), 19.8% indicated that it partially met, 16.3% considered that the course exceeded their expectations, and no participant answered “did not meet the expectations.” Figure 2 describes the results regarding participants’ expectations.

Fig. 2.

Participants’ expectations about the training course.

Level 2: Skills and Learning

Pre-post analysis revealed that self-perceived knowledge was significantly higher immediately after the training course (median 4, high knowledge; IQR 1) than before the training (median 3, moderate knowledge; IQR 1) (z = −6.778, p < 0.001) (Wilcoxon signed-rank test) (Table 3). One participant did not answer this question at baseline (n = 100).

Table 3.

Self-perception of knowledge at baseline and post-training evaluation

Categories	Baseline, n (%)	Post-test, n (%)
No knowledge	8 (7.9)	0 (0.0)
Poor knowledge	24 (23.8)	0 (0.0)
Moderate knowledge	53 (52.5)	24 (23.8)
High knowledge	13 (12.9)	57 (56.4)
Full knowledge	2 (2.0)	5 (5.0)

There is a missing value at baseline.

The analysis of the 24 multiple-choice questions used to measure participants’ knowledge/learning after the completion of training revealed that most participants (87.2%) had from 51 to 100% of correct answers. The median percentage of correct answers was 70.8% (IQR 21.0).

The mean DAS score was 116.4 (SD 9.9) at baseline and 121.9 (SD 11.4) after the completion of training, showing an increase of 4.7%. This increase is represented in Figure 3. There was a significant difference pre-post (t [84] −4.853; p < 0.001), which means there was an improvement in the participants’ attitudes toward dementia.

Fig. 3.

Comparison of attitudes toward dementia at baseline and post-training evaluation. The boxplot shows Dementia Attitude Scale (DAS) median scores (baseline Med = 117, IQR 15; post-training Med = 123, IQR 17); higher scores indicate positive attitudes towards dementia.

Level 3: Behavior Changes

Following the training course, most participants said that their behavior or practice changed (n = 73, 91.3%) as a result of their attendance. Thematic analysis of open responses on behavior/practice change revealed four themes:

• 1.Care provision and interaction with PwD – participants reported improvements in the care provided, increased use of a person-centered approach in their daily practice, and the adoption of a holistic or multidisciplinary approach in dementia. Also, the quality of interactions between staff and PwD improved after the training, according to their views:
  • •“I understood the importance of providing care ‘with’ instead of ‘for’ PwD and the importance of the respect and the integration of PwD’s life history, always recognizing their values and preferences.” (healthcare assistant)
  • •“I started to dedicate more time to promoting patient autonomy, especially during meals.” (healthcare assistant)
  • •“(There is) better adequacy of care to the singularities of the PwD.” (nurse)
  • •“(I gained) a broader vision about the patient and about the disease as a whole.” (doctor)
• 2.Communication – the training improved the communication skills of participants, as explained by a healthcare assistant and a nurse:
  • •“I have learnt how to communicate and interact adequately with the environment around me.” (healthcare assistant)
  • •“(I have improved) the communication with the patient.” (nurse)
• 3.Family and caregivers – the training was also perceived as useful for a deeper recognition of the negative impact of the disease on the family and caregivers and the recognition of the importance to include the family/caregivers in the care attention. In the words of a healthcare assistant, the course helped:
  • •“(…) to recognize that the loss of autonomy or the alienation of reality has consequences for several caregivers and families.” (healthcare assistant)
• 4.Self-confidence – the training contributed to increasing confidence among staff in the care they provide, as stated by healthcare assistants:
  • •“(I improved) the approach with PwD. I feel more confident.” (healthcare assistant)
  • •“(…) I have more facility in proving care to the patient.” (healthcare assistant)

Discussion

This paper presented the evaluation of the impact of an online pre-service dementia training course designed for professionals and volunteers who will work with PwD. To the best of our knowledge, this study is the first to measure the impact of one dementia training course in a specialized center based on the Kirkpatrick’s Evaluation Framework (2009) targeting a multidimensional team of operational and executive professionals. Most of the studies on this topic evaluated the impact of dementia training in a hospital setting [4–6].

Our training course followed international recommendations, such as the recruitment of experienced facilitators from different professions (multidisciplinary training) and the duration of eight or more hours in total [17]. The training was developed before the opening of the center, following WHO [2] recommendations for a public health response for dementia. WHO recognizes the role of pre-service training of all cadres of the health and social workforce in order to ensure that those who are meant to serve PwD and their caregivers are equipped with the knowledge they need. This is particularly important for professionals with low levels of literacy or education.

Also, a note on the delivery method of the training course: COVID-19 pandemic circumstances required the course to be delivered online, but there is evidence that face-to-face methods increase the effectiveness of the training [18]; this is one point to improve in the future.

Usually, care institutions that provide health or social support to PwD have a small number of professionals with specialized training in dementia which is an essential pre-requisite to guarantee high quality of care [4]. Our center was no exception; for 64% of participants, this course was their first training or education program about dementia. This result is similar to the obtained in others studies. Elvish et al. [4] reported that 52% of participants in the “Getting to Know Me” program, performed in one hospital in the UK, had received no prior training in dementia care. Further, Schneider et al. [19] obtained a higher absence of prior training focusing on dementia among participants in a 2-day dementia training program performed in six hospitals in Germany with 91.5% without prior training.

Our course had a positive impact in all dimensions evaluated, following Kirkpatrick’s evaluation framework [12, 13]. First, the positive reactions (level 1) of the participants highlight the adequacy of the training content and the methods of delivery for this group of participants. Others studies analyzed the reaction of the participants regarding dementia training courses. One example is O’Brien et al. [20] that developed a pre-post study to analyze the impact of a 2-day dementia communication skills training course for healthcare professionals that worked regularly with PwD. The expectations about the course were assessed using a scale from 1 to 10 and the results indicated that almost all participants (95%) found the course met their expectations. This is an important point – the training can be more effective if it meets the expectation of participants.

Secondly, the evaluation of the training in this study showed good learner knowledge gains (level 2). However, these results are lower than the obtained in others studies. Hobday et al. [6] analyzed the knowledge obtained by participation in the CARES Dementia-Friendly Hospital Program among 25 participants with the Dementia Care Knowledge (a 19-item measure with multiple-choice and true/false items). Results showed on average 91.6% (SD 6.08) correct answers at post-training evaluation. However, as there was no pre-training evaluation of participants’ knowledge, it is unclear how much the training itself explains the learning. We highlight that successful knowledge learning should also consider the ability to retain knowledge over a long period. It will be interesting to explore this issue in future studies.

In our study, the professionals’ attitudes toward PwD improved, which supports the existing literature [8, 11]. The results by Schneider et al. [19] indicated that the dementia training program “PwD in the dementia training program” (six 2-day training blocks on dementia) had a positive effect on the perceptions and attitudes of hospital staff toward PwD and also increased participants’ knowledge regarding PwD. Also, Banks et al. [21] implemented a blended learning dementia champions program comprising five study days alongside a half day in a community care setting, with 100 working staff. The authors found that participants showed a significant positive change in attitudes toward PwD as assessed using a validated measure (Approaches to Dementia Questionnaire [22]).

Lastly, in our study, nearly all participants reported behavior changes, towards a better professional practice, as a result of the training. These results can be associated with higher self-confidence to care for PwD, as an output from the training, as suggested by the qualitative analysis. However, this output should be confirmed quantitatively in further research.

These new insights regarding dementia training show that our study is innovative. Furthermore, the quasi-experimental design and the use of the Kirkpatrick’s evaluation framework to identify what works in dementia care training [13] strengthen the study. However, this evaluation framework has some limitations discussed in the literature. Holton [23, 24] flagged other factors may influence the training outcomes, such as the organizational climate and motivation to learn, which we did not capture. Our study has also limitations. First, only 86 of 101 participants completed the post-training evaluation and is unknown the level of satisfaction, learning, and attitudes toward dementia of the 15 participants who did not complete the evaluation. We compared both groups, participants with (n = 86) and participants without post-training evaluation (n = 15) regarding age, level of education, previous training on dementia, expectation towards the training course, previous knowledge about dementia and attitudes, and no statistical differences were found. For this reason, we believe that the reasons for not completing the questionnaire may be related to a lack of interest or time. A larger sample size would enable more statistical power, although the number is representative of the staff working at the specialized center (in total now 103 workers). Furthermore, limitations of self-reported measures, in general, have to be considered, such as social desirability in the self-evaluation procedures. Finally, this study was run before the center opening, which limited the range of the variables we could measure, such as the impact of training on outcomes for PwD and their family members (level 4 of Kirkpatrick’s evaluation framework). This is something we intend to measure in the near future. Also, the study only reported the immediate impact of the training course. To explore the sustainability of the training, data from follow-up time points will be necessary.

In conclusion, even though dementia is recognized by the WHO as a public health priority [2], the care provided often does not meet standards of person-centered and best practice approaches [25]. Such gaps in care suggest the need for increased and enhanced training to improve skills, knowledge, attitudes, and behavior among professionals. The present study contributes to the effort to improve the quality of care of PwD, providing evidence about an effective first-level training course directed to all professionals who work with PwD. This study could be helpful to administrators, clinicians, and/or researchers who may be interested in developing and delivering a similar training program in institutions that provide care to PwD and their families.

The positive results obtained in the study suggest the course was an effective way of providing first-level dementia training to staff who work with PwD in our center. The model could be transferable to others centers dedicated to dementia care in similar settings. A second-level training (intermediate course) is being prepared and delivered in the future.

Acknowledgments

The authors thank all participants and lecturers of the training course, as well as the institution’s administration for supporting the study.

Statement of Ethics

Our research complied with the guidelines for human studies and was conducted in accordance with the World Medical Association Declaration of Helsinki. Written informed consent from all participants in this study was obtained for participation. The study protocol was reviewed and approved by the Ethics Committee of the Santa Casa da Misericórdia de Riba D’Ave (approval number 003/21).

Conflict of Interest Statement

Authors have a financial relationship with Santa Casa da Misericórdia de Riba D’Ave, CIDIFAD – Centro de Investigação, Diagnóstico, Formação e Acompanhamento das Demências, in form of an employment contract.

Funding Sources

This research received support from the North Portugal Regional Operational Program (part of PORTUGAL 2020 Partnership Agreement) that supports Santa Casa de Misericórdia de Riba D’Ave through the operation n° NORTE-06-3559-FSE-000175. Also, this work is financed by national funds through FCT Fundação para a Ciência e a Tecnologia, I.P., within the scope of the project “RISE – LA/P/0053/2020” and by “UIDP/04255/2020”.

Author Contributions

N.D. was responsible for the data analysis and interpretation and for drafting the article. S.A. and B.G. were involved in the original conception, and design, and contributed to analysis and interpretation. All authors reviewed and approved the manuscript.

Funding Statement

This research received support from the North Portugal Regional Operational Program (part of PORTUGAL 2020 Partnership Agreement) that supports Santa Casa de Misericórdia de Riba D’Ave through the operation n° NORTE-06-3559-FSE-000175. Also, this work is financed by national funds through FCT Fundação para a Ciência e a Tecnologia, I.P., within the scope of the project “RISE – LA/P/0053/2020” and by “UIDP/04255/2020”.

Data Availability Statement

The data that support the findings of this study are not publicly available due to their containing information that could compromise the privacy of research participants but are available in an anonymized form from the corresponding author (Natália Duarte) upon reasonable request.