Practicality Assessment of the Improved ICT-Based Dementia Care Mapping Support System

Abstract

Objectives:

Authors have developed an ICT (Information-Communication Technology)-based Dementia Care Mapping (DCM™) support system and assessed its usefulness previously using videos. The aim of this study is to evaluate practicality of the improved support system by actual mapping.

Methods:

We obtained highly reliable mapping data by a team of 3 mappers (M1, M2, and M3), who mapped 5 elderly people with dementia continuously over 4 hours in 2 sites. M1 mapped by conventional paper-based DCM™, while M2 and M3 mapped using the support system. Collected mapping data (n = 8, total of 232.5 hours = 2790 time frames) were compared.

Results:

The concordance rates between M1 and M2/M3 were over 70%; the required level for DCM™ evaluation purposes, with 1 exception. We also obtained users’ positive ratings regarding the system usability.

Conclusion:

Practicality of the ICT-based DCM™ support system was confirmed for evaluation purposes, demonstrating the system is ready for practical use.

Introduction

Dementia Care Mapping (DCM™ ) ^1,2 was developed aiming at implementing Person-centred Care (PCC) ³ in formal dementia care facilities. Dementia Care Mapping is a developmental evaluation system through which care quality is observed and fed back to care staff team for developing better care practice via its repetitive cyclic process (Figure 1). The feature of DCM™ is to promote an understanding of care practice from the viewpoint and personal experience of people with dementia, ^3,4 which is represented by observers (mappers). Hence, mappers play a very important role in DCM™.

Figure 1.

General framework of DCM™, the developmental evaluation system. DCM indicates Dementia Care Mapping.

Dementia Care Mapping is a well-developed method, which is standardized in the United Kingdom ⁵ and has been used successfully worldwide for over 20 years. ^{6 -10} However, recent randomized controlled trials (RCTs) of DCM™ have reported mixed results on efficacy and indicated that process and implementation issues are potential factors to explain the differences in the results. ^{11 -13} To explore the issues, Surr et al ¹⁴ conducted a systematic review and found out that, from the relatively little evidence available on how the approach was applied in practice, appropriate manager support, and adequate staffing seemed to be essential to the successful implementation of DCM™. They also conducted an RCT for evaluating the effectiveness and cost-effectiveness of DCM™ based on the study protocol. ¹⁵ During the process evaluation conducted as part of the RCT, Griffiths et al ¹⁶ identified the barriers and facilitators to DCM™ implementation at the mapper level (eg, motivation and confidence), the intervention level (eg, understanding of DCM™), and the care home level (eg, staffing issues, manager support). It was highlighted by these studies that good understanding and strong support by managers and staff were quite important for successful implementation of DCM™. Also highlighted were the complex nature of DCM™ and requirement for high skills and qualities of the mapper role. Griffiths, therefore, suggested alternative models of implementation such that external teams of expert practitioners and/or researchers undertake the mapper role, if appropriate ¹⁶ .

It may be a safe approach toward successful implementing of DCM™ to rely on external professional teams; however, it might restrict the implementation of DCM™ if such professional teams are not sufficiently available. In addition, the authors often hear from many of the domestic mappers that in-house mappers can learn a lot from mapping at external care facilities in particular, by sharing the mapping experiences and cultures of both sides with the staff members there. Thus, care home staff members can learn from mapping not only inside but also outside the own care home as in-house mappers, which may be one of the big advantages of implementation of DCM™ without relying on external professionals and the motivation for staff members to undertake the mapper role instead of their being full of demanding job. Hence, we think it is more important and sound assisting as many in-house staff members as possible in undertaking the role of the mapper or acquiring the mapper’s viewpoint than relying on external professionals. The most complex process involved in DCM™ is the observational method, and its complexity is likely to account for posing issues at both the mapper and the intervention levels which Griffiths identified. ¹⁶ We think it is possible to solve the complexity by utilizing technology; thereby mappers can make the best use of their ability of observing people with confidence, without concerning for the complex method.

Hence, we have developed an ICT-based DCM™ support system on a tablet PC (Figure 2) ¹⁷ in order to reduce mappers’ burden, making it easier to carry out DCM™, and assessed its usefulness using videos. ¹⁸ As a result, we obtained users’ positive ratings of the system for burden reduction and its usability. Moreover, the developed prototype system indicated to have already reached the level of conventional paper-based DCM™ in terms of mapping accuracy.

Figure 2.

Graphical user interface of the developed DCM™ support system with handwritten notes.  DCM  indicates Dementia Care Mapping.

In this article, we report on the practicality of the improved support system by actual mapping in dementia care facilities. This study is based on   DCM   8; the latest edition of DCM™ published in 2005. ² To the best of our knowledge, there is no research published to date concerning the development of ICT-based DCM™ support systems except our studies. ^17,18

Methods

Improvement of the ICT-Based Support System for DCM™

The DCM™ observational method is described elsewhere. ^6,19 Because the method is rather complex, the descriptions tend to be wordy, such as an observer (mapper) observes 5 elderly people with dementia (participants) continuously over a representative period (eg, 6 hours during the waking day) in communal areas of care facilities. After each 5-minute period (a time frame), 2 types of codes are used to record what has happened; the behavior category code (BCC) and the mood and engagement value (ME value). The BCC describes 1 of 23 different domains of participant behavior. The ME value describes the participant’s mood state alongside their level of engagement with their environment, being expressed on a 6-point scale ranging from −5 to +5 and so on and so forth. Since it is not easy for general readers to understand the complex method described by words, the authors made schematic explanations ¹⁸ illustrating a clear picture of DCM™ and its complex nature at a glance; the observation and feedback process of DCM™ is shown in Figure 3A, and the manner of deciding the representative BCC and ME value for each time frame according to the operational rules ² is illustrated in Figure 3B.

Figure 3.

Schematics of mapping and feedback process and the operational rules of DCM™. A, Observation and feedback process of DCM™. A mapper observes the participants’ activities and their state of well/ill-being, using (i) the BCC and (ii) the ME value. After each 5-minute period (time frame), a mapper decides a representative pair of BCC and ME value according to (iii) the DCM™ operational rules for record. The recorded time-series data look like (iv) a resultant chart. A mapper also records qualitative notes concerning (v) PDs and PEs, which are the staff behaviors having potential to undermine/uphold the 5 psychological needs of people with dementia. Finally, a mapper analyzes the results to prepare (vi) a summary for giving feedback to the care staff team ² .   DCM   indicates Dementia Care Mapping; PwDementia, the person with dementia (the participant of DCM). B, Decision-making of the representative BCC and ME for each timeframe according to the DCM™ operational rules. When 2 or more behaviors occur within 1 time frame, follow the operational rules to decide. They are: (1) BCCs of higher potential, for positive ME, take precedence over those of lower potential, (2) record the behavior with the longest duration, (3) record the behavior with the most extreme ME value, and (4) record the behavior engaged in for the latter part of the time frame. The rules are applied one at a time beginning with (1) through (4) until a decision can be made ² . BCC indicates behavior category code; ME value, mood and engagement value; PD, personal detraction; PE, personal enhancer.

Dementia Care Mapping is a well-developed method. However, it sets high requirements on the mapper in order to ensure its quality. The mapper needs to keep track of time precisely during 6-hour period to record time frame results every 5-minute according to the rules, which is difficult and burdensome. The ICT-based DCM™ support system helps reduce the mapper’s burden ¹⁷ in such a way that the system keeps time and automatizes the rules for the mapper to simply record what is observed at a given moment. The mapper selects a name of the participant (elderly person with dementia for observation) on the entry form of the graphical user interface (GUI), shown in Figure 2, enters observed codes of BCC and ME by tapping appropriate icons, and handwrites notes as necessary. Then the system processes the entered codes automatically to finalize the representative (BCC, ME) pair for each time frame according to the operational rules. Thus, the mapper can concentrate on observation, without paying attention to keeping track of time and the complex rules. Regarding the major functionality and specifications of the DCM™ support system, please refer to our previous article. ¹⁸

We actually obtained quite positive ratings of the system for burden reduction and its usability in the previous assessment ¹⁸ using role-play videos of 2 participants, except for one problem that cannot be left unsolved. You need to select a proper name-tab to switch the entry form for the intended participant through the GUI; however, not a few mappers had forgotten or mistaken to select a proper one in the previous experiment. In this study, the number of target participants increased up to 5 from the previous 2; therefore, the mistake in selecting a proper name-tab would cause a more serious problem. In order to solve the issue, we changed the main screen of the GUI so that you can now enter BCCs and/or ME values for any one of the 5 participants without switching the entry form, as shown in Figure 4A. The sub-screen of each participant appears by just 1 tap from the main screen, as shown in Figure 4B, and you can review the entered records in detail or make handwritten notes.

Figure 4.

Graphical user interface of the improved DCM™ support system. A, Main screen of the DCM™ support system; entry form of BCC and ME for 5 participants. B, Sub-screen for each participant, to review mapping records and/or enter handwriting notes. BCC indicates behavior category code; DCM, Dementia Care Mapping; ME, mood and engagement.

We made other improvements, such as component bar charts on the main screen to indicate changes of each participant’s BCC and ME every 15 seconds (Figure 4A), a side window on the sub-screen (Figure 4B) for reviewing record of any time frame backward/forward, and so on. You can also print out mapping results in a tabula worksheet form (Figure 5), which includes the rules applied to decide the predominant BCC and ME value for each time frame. Printouts are quite useful to review and make amendments after mapping as you can look at several sheets of paper at a glance.

Figure 5.

Printout of conventional-style raw data sheet with time stamp records and handwriting notes.

Data Collection

When we assessed the usefulness of the support system previously, ¹⁸ we collected mapping data under the same condition using videos and compared them with prepared standard answers. Real mapping at dementia care facilities, on the contrary, is quite different since conditions vary depending on places, time, participants, staff, and so on, all of which are uncertain. Above all, there are no correct answers for real mapping, therefore, it is a difficult issue of both how to collect and evaluate data. It may be reasonable only to compare reliable data. Hence, we tried to collect as highly reliable data as possible by recruiting high-level mappers, as described later in the Subjects section.

The data were collected as follows:

1. Prime Map: In order to obtain IRR (inter-rater reliability) analyzing data, a team of 3 mappers (subjects) mapped 5 participants (elderly persons with dementia) in common continuously over 4 hours (up to 6 hours). One of them (M1) mapped by conventional paper-based DCM™, while other 2 (M2 and M3) mapped using the support system.

2. Prep Map: Prior to the Prime Map, all 3 subjects carried out mapping by conventional DCM™ for 1 to 1.5 hour, in order to obtain baseline data to check IRR concordance rate.

3. After the Prime Map, we conducted a questionnaire survey regarding the usefulness and usability of the improved support system.

Subjects, Participants, and Sites

We set the following 2 recruitment criteria in order to obtain highly reliable data:

1. Those who achieved correctly answered rate above 80% in the previous video map.

2. Those DCM™ users of higher level (trainer or advanced user of DCM™) who conduct mapping in a regular basis.

We recruited 3 mappers for each criterion 1 and criterion 2 and conducted tests at 2 different nursing homes (case 1 and 2, thereafter).

Notes: Each mapper M1, M2, and M3 in case i will be noted as M1_i, M2_i, and M3_i, respectively.

In addition to reliable data, feedback from ordinary users regarding the usability of the system is inevitable for practicality assessment. We, therefore, recruited a basic DCM™ user who was not so experienced in mapping and conducted a set of Prime Map with one of the authors who is an advanced user of DCM™ (case 3).

Characteristics of the subjects, participants, and test sites are listed in Table 1.

Table 1.

Characteristics of Subjects Patients, Participants, and Sites.

Demographic	Characteristic	n	(%)
(a) Subjects Patients		8
Gender	Male	6	(75)
	Female	2	(25)
Age	30	1	(12.5)
	40	3	(37.5)
	50	4	(50)
Role	Care profession	3	(37.5)
	Management	2	(25)
	Doctor	1	(12.5)
	Training and education	2	(25)
Level of DCM Training	Basic user	2	(25)
	Advanced user	5	(62.5)
	Higher level user	1	(12.5)
Computer literacy	High	2	(25)
	Normal	2	(25)
	Poor	3	(37.5)
	Illiterate	1	(12.5)
(b) Participants		15
Gender	Male	3	(20)
	Female	12	(80)
Age	60	1	(6.7)
	70	3	(20)
	80	8	(53.3)
	90	3	(20)
(c) Sites		3
Type	Nursing home	2	(66.7)
	Day center	1	(33.3)

Abbreviation: DCM, D dementia C care M mapping.

Practicality Evaluation of the Support System

In DCM™, each pair of mappers should achieve at least 70% concordance in order to map for evaluation purposes. ^2,20 It is therefore necessary that, in cases 1 and 2, concordance rates CR(M1, M2) and CR(M1, M3) should achieve at least 70% concordance in order for utilizing the support system to be good for evaluation purpose, where CR(Mj, Mk) denotes the concordance rate in Prime Map between Mj and Mk. Moreover, if CR(M2, M3), the concordance rate between the 2 mappers both used the support system, achieved more than 70% at the same time, then the reliability of the system would be confirmed to a certain degree.

If CRs resulted in lower than 70%, we would need to analyze the data and discuss the results based on the CR0s, where CR0 is the baseline IRR concordance rate in Prep Map.

Regarding case 3, we calculate CR between the basic user and the advanced user for reference, though the level of reliability of the data obtained by the basic user is unknown.

Usefulness Evaluation of the Support System

We evaluated the usefulness of the support system likewise as previously, ¹⁸ looking at the decreased level of mapping difficulty using the support system in cases 1, 2, and 3, as follows:

• We formerly investigated mappers’ difficulties and/or burden being experienced during mapping ¹⁸ as listed in Table 2.

Table 2.

Mappers’ Difficulties and/or Burden during Mapping.

Difficult items encountered during mapping
1. Difficulty about operational rules
BCC	① Choose higher potential BCCs when 2 or more occur in one time frame
	② Keep track of time accurately to record the BCC of longest duration
	③ Record the BCC with the most extreme ME value, if ① and ② are the same
	④ Record the BCC of latter part of time frame, if ① through ③ are the same
ME	⑤ Choose from limited ME values properly for [B C N U W]
	⑥ Allocate ME value properly from M and E values based on the 3 rules
	⑦ Keep track of time accurately to choose the predominating ME value
Other	⑧ Mark “*” in the sixth time frame when uninterrupted negative ME continues
	⑨ UNME (uninterrupted negative ME) is not interrupted by [N]
2. Difficulty about observation and recording
Timekeeping	⑩ Keep track of time accurately to recognize time frame every 5-minute interval
Recording	⑪ Do not overlook observing when recording time frame results every 5 minutes
	⑫ Decide representative BCC correctly when various BCCs occur continuously
	⑬ Decide representative ME correctly when M and E change continuously
Taking notes	⑭ Take notes at the right moment, do not lose the chance to take notes
	⑮ Make easy-to-read notes; time and timing, order of occurrence, duration, and so on
Correction	⑯ Correct and/or add codes or notes during mapping
	⑰Correct data/info on raw data sheet for report writing after mapping

Abbreviations: BCC, behavior category code; ME, mood and engagement.

Ask a subject, concerning each item in Table 2, their usual feeling of level of difficulty (D) and level of help (H) that they have felt after using the support system:

(D) = {-3: difficult,-1: somewhat(s.wh.) difficult, +1: s.wh. easy, +3: easy}

(H) = {-3: unnecessary,-1: s.wh. unnecessary, +1: s.wh. helpful, +3: helpful}.

• Calculate the obtained easiness level (E) by the support system by Equation (1):

$$\left(\text{E}\right)\text{ = }\left(\text{H}\right)\text{ – }\left(\text{D}\right)$$ 1

where −6≤ (E) ≤6, and the more (H) exceeds (D), the larger (E) becomes.

Calculate the obtained easiness level (E(j)i) concerning each item j (j = 1-17) for each subject i (i = 1 to N, N: total number of subjects). If mean (E(j)i) > 0, then item j is considered to have become easier by the support system, where mean (E(j)i) is the average of (E(j)) of all subjects.

Measures

DCM™ Codes. The BCCs were assigned to participants (elderly people with dementia) along with the ME values, both of which were recorded as per DCM 8 User’s Manual. ²

Ethical Issues

Ethical approval was granted by the Human Ethics Committee of Graduate School of Engineering, Kobe University. We explained to all of the subjects that subject anonymity would be preserved strictly, and written informed consent was obtained prior to conducting each experiment. In addition, the director of each care facility had explained all of the mapping participants about the experiment and obtained consent for their participation, and we obtained written informed consent from the directors.

Results

Practicality Evaluation

CR0s; the concordance rates in Prep Map in cases 1 and 2, and CRs; the concordance rates in Prime Map in cases 1, 2, and 3, are listed in Table 3. Behavior category code and ME distributions were observed relatively broad in any of the cases, which are also included in Table 3.

Table 3.

Concordance Rates and Code Distributions of Mapping in 3 Cases.

Map cases	Concordance rate and code distribution
Case 1: Nursing home A	Mapper pair for concordance analysis
	M11-M21	M11-M31	M21-M31
Prep Map (period = 1.5 h)
Relevant time frames	144	144	144
Concordance rate: CR0 (%)	72	71	74
BCC distribution: Code (%)	13 BCCs: [A(22.5), D(21.1), E(0.9), F(7.3), I(5.0), K(0.9), L(8.7), O(9.2), P(4.6), V(0.9), X(1.4)]; B(5.0); N(12.4)
ME distribution: value (%)	+1 (82.7); +3 (17.3)
Prime Map (period = 4.5 h)
Relevant time frames	254	276	290
Concordance rate: CR (%)	74	74	83
BCC distribution: code (%)	11 BCCs: [A(26.6), D(13.3), F(28.8), K(3.4), L(9.0), O(2.4), P(3.9), V(2.4), X(2.8)]; B(3.9); N(3.6)
ME distribution: value (%)	−1 (1.1); +1 (84.8); +3 (14.1)
Ratio of concordance rates: CR/CR0	1.03	1.04	1.13
	Mapper pair for concordance analysis
Case 2: Nursing home B	M12-M22	M12-M32	M22-M32
Prep Map (period = 1 h)
Relevant time frames	120	120	120
Concordance rate: CR0 (%)	58	73	68
BCC distribution: code (%)	11 BCCs: [A(8.9), D(18.9), F(10.0), K(4.4), L(26.1), P(2.2), V(3.9), Y(1.7)], B(11.7), N(10.0), [C(2.2)]
ME distribution: value (%)	-1 (3.7), +1 (84.0), +3 (12.3)
Prime Map (period = 5 h)
Relevant time frames	568	558	580
Concordance rate: CR (%)	66	70	73
BCC distribution: code (%)	15 BCCs: [A(11.7), D(16.7), F(9.9), G(0.1), J(0.2), K(6.9), L(20.0), P(2.6), T(0.1), V(9.0), X(1.7), Y(0.3)], B(10.5), N(10.0), [C(0.2)]
ME distribution: value (%)	-1 (4.1), +1 (77.4), +3 (18.2), +5 (0.3)
Ratio of Concordance rates: CR/CR0	1.13	0.97	1.08
	Mapper pair for concordance analysis
Case 3: Day center C	M23-M33
Relevant time frames	474
Concordance rate: CR (%)	66
BCC distribution: code (%)	14 BCCs: [A(14.6), D(7.3), E(7.5), F(10.3), J(6.5), K(2.0), L(23.1), O(1.0), P(2.4), V(2.4), W(0.2), X(5.3)], B(8.7), N(8.9)
ME distribution: value (%)	+1 (70.3), +3 (27.8), +5 (1.95)

Abbreviations: BCC, behavior category code; ME, mood and engagement.

In case 1, CR(M1₁, M2₁) and CR(M1₁, M3₁) were both 74%, and CR(M2₁, M3₁), the concordance rates between mappers both used the support system, reached as high as 83%. For CR0(M1₁, M2₁), CR0(M1₁, M3₁), and CR0(M2₁, M3₁), the baseline concordance rates in Prep Map were 72%, 71%, and 74%, respectively, all of which were above 70%. The ratio of concordance rate, CR/CR0, for each pair of subjects was not less than 1.0, as shown in Table 3, indicating there were no signs of deterioration using the support system.

In case 2, contrary to case 1, not all CRs reached 70%; CR(M1₂, M2₂), CR(M1₂, M3₂), and CR(M2₂, M3₂) were 66%, 70%, and 73%, respectively. The concordance rates in Prep Map were even lower; CR0(M1₂, M2₂) in particular was only 58%. Concordance rates per participant in case 2 are shown in Table 4. Most of the concordance rates reached around 70%, except for those regarding participant 2 (P2). All of the concordance rates regarding P2 were quite low; CR0(M1₂, M2₂), CR0(M1₂, M3₂), and CR0(M2₂, M3₂) were 21%, 54%, and 25% and CR(M1₂, M2₂), CR(M1₂, M3₂), and CR(M2₂, M3₂) were 50%, 61%, and 65%, respectively. Those rates regarding P4 were also slightly lower than others.

Table 4.

Concordance Rates Per Participant in Case 2 Mapping.

Mapper pair and map category		Concordance rate per participant
		P1	P2	P3	P4	P5	Total
	M12-M22
Prep Map (1 hour)	Relevant time frames	24	24	24	24	24	120
	Concordance rate: CR0 (%)	67	21	71	58	75	58
Prime m Map (5 hour)	Relevant time frames	114	106	116	116	116	568
	Concordance rate: CR (%)	82	50	67	66	65	66
	M12-M32
Prep Map (1 hour )	Relevant time frames	24	24	24	24	24	120
	Concordance rate: CR0 (%)	71	54	88	67	83	73
Prime m Map (5 hours)	Relevant time frames	114	106	116	106	116	558
	Concordance rate: CR (%)	76	61	75	77	60	70
	M22-M32
Prep Map (1 hour)	Relevant time frames	24	24	24	24	24	120
	Concordance rate: CR0 (%)	75	25	83	79	75	68
Prime Map (5 hours)	Relevant time frames	120	112	120	108	120	580
	Concordance rate: CR (%)	81	65	74	69	74	73

The BCC and ME distributions of P2 observed by the 3 subjects, M1₂, M2₂, and M3₂, are shown in Table 5. Differences per subject were observed both in BCC and ME; [N, D, C] and [+1, −1] in Prep Map, and [N, D, K] and [−1] in Prime Map.

Table 5.

Behavior Category Code and ME Value Distributions of Participant P2 per Mapper in Case 2.

Map category	Mapper	BCC									ME value
		A	B	C	D	F	G	K	N	P	−1	+1	+3
Prep Map	M12	1	0	0	0	2	0	0	9	0	0	2	1
(1 hour)	M22	0	0	0	7	2	0	2	1	0	0	11	0
	M32	1	0	4	0	1	0	1	4	1	4	3	1
Prime Map	M12	2	3	0	6	8	0	10	23	1	0	28	2
(5 hours)	M22	0	0	0	23	7	1	20	4	1	17	30	5
	M32	0	0	2	16	8	0	21	8	1	2	39	6

Abbreviations: BCC, behavior category code; ME, mood and engagement.

In case 3, CR(M2₃, M3₃), the concordance rate between a basic user and an advanced user of DCM™, both used the support system, was 66%.

Usefulness Evaluation

We calculated obtained easiness level (E) for each item listed in Table 2, estimated by Equation (1), using the data obtained by the 6 subjects who used the support system in cases 1 through 3. We had previously obtained values of (E) from 22 mappers the same way after the video map. ¹⁸ In order to compare the 2 sets of data, the data obtained by 4 high-level mappers in the previous assessment and those by 5 subjects in current assessment, excluding the basic user in case 3, were averaged and shown in Figure 6 as “Easiness_v” and “Easiness,” respectively. Also shown in Figure 6 were “Difficulty_v” and “Difficulty,” for reference, which are twice of the negated level of difficulty (D) in previous and current assessment, respectively. Items 1, 8, 16, and 17 in Table 2 were eliminated from Figure 6 because those items were answered easy for high-level mappers.

Figure 6.

Obtained easiness level (E) of high-level mappers after previous video map and prime map. Difficulty is indicated as “–2(D)”; twice of the negated level of difficulty (D), for reference. Difficulty_v and Easiness_v, data obtained after the previous video map; Difficulty and Easiness, data obtained after prime map.

The obtained easiness level (E) of current assessment looked more or less the same and was all positive as the previous one, with one exception of item 14, to take notes at the right moment and/or not to lose the chance to take notes, which received a very low point for obtained easiness level. In addition, we received responses to open question regarding item 14 as follows:

• It is not easy to make notes in relation to code entries, since the entry forms for notes and codes are separated. It is easier to write on a sheet of paper.

• The support system is superior and easy for entering codes, however, if only it were more user friendly regarding notes taking.

• It is a bit hard to enter codes or take notes when 2 or more participants become involved in some activities at the same time. It is also true with the paper-based system, though.

• It is impossible to enter codes/notes timely when observing many participants.

• You must switch entry-form screens back and forth when 2 participants are talking with each other and another participant begins some activity, and so on. It would not be a problem if I got used to operating the system, though. I may be able to cope with it using the common notes entry form; I noticed the entry form available after the mapping.

Discussion

Practicality of the Support System

The observed BCCs and MEs showed a relatively wide distribution in all cases 1–3 (Table 3), in which there was no bias that made obstacle to evaluate IRR concordance.

In case 1, all of each pair of subjects (mappers) achieved over 70% concordance in Prep Map, therefore, it was assured that the data obtained by the 3 subjects would be valid for evaluation purposes. And the concordance rates CR(M1₁, M2₁) and CR(M1₁, M3₁) in Prime Map also reached higher than 70%, thus it was confirmed that the support system could be utilized for evaluation purposes without any problems. Furthermore, the fact that CR(M2₁, M3₁) achieved 83% indicates the reliability of the support system to a certain extent.

In case 2, on the other hand, the concordance rate CR(M1₂, M2₂) in Prime Map did not reach 70%, indicating that at least the pair of mapping data obtained by M1₂ using conventional DCM™ and by M2₂ using the support system did not seem to be valid for evaluation purposes. We should note, however, that CR0(M1₂, M2₂) in Prep Map was as low as 58% and CR0(M2₂, M3₂) did not reach 70%, despite the fact that the 3 subjects were mappers of high-level (trainer or an advanced user). In order to verify what was happening in case 2, we analyzed the baseline data of the 3 subjects M1₂, M2₂, and M3₂.

As shown in Table 4, concordance rates regarding participant 2 (P2) were all quite low in any pair of subjects. There were big differences among the codes recorded for P2 per subject, as shown in Table 5, resulting in the low concordance rates for P2, which seemed to be the major reason for lowering the total concordance rates. Hence, we looked into the raw mapping data, including notes, of the 3 subjects regarding P2 in detail and found out why and how the differences have occurred, as follows:

• All of the 3 subjects made notes regarding subtle body movements of the participant 2 more or less the same as follows:

  P2, who can stroll with a wheelchair, was dozing fairly long time during the mapping period, laying back on her wheelchair with her head putting down. Every once in a while, she slightly moved back and forth, gave a slight shrug, or leaned forward.

• M1₂ recorded BCC = [N] (sleeping, dozing) for as many as 9 time frames (Table 5), presumably because:

  M1₂ made notes of P2’s slight moving back and forth and yet recorded [N] for the particular behavior, taking the motion as part of dozing. P2 was observable just from behind from where M1₂ was seated; therefore, M1₂ probably could not recognize P2's expression or behavior in detail.

• M2₂ recorded BCC = [D] (Self-care) for as many as 7 time frames, presumably because:

  M2₂ was seated to M1₂’s immediate left, being able to observe P2’s behavior a little more in detail. And for the 9 time frames in which M1₂ recorded [N], M2₂ recorded 6[D]s + 2[K]s (Walking, standing or moving independently) + 1[N]. M2₂ apparently took P2’s motion of adjusting her posture as [D], and the slight rocking motion of the wheelchair as [K], both of which as P2’s intentional body motion apart from [N].

• M3₂ alone recorded BCC = [C] (Being disengaged, withdrawn) with negative ME for 4 time frames, presumably because:

  M3₂ was seated to M2₂’s immediate left likewise, being able to observe even P2’s expression a little. And for the 9 time frames in which M1₂ recorded [N], M3₂ recorded 4[C]s + 1[K] + 4[N]s. When P2 leaned forward, M3₂ noticed an expression of pain, which was scribbled down, and recorded BCC = [C] and ME = [−1] (“Small signs of negative mood”). Regarding the behavior of slight back and forth motion, M3₂ recorded [N] taking it as part of dozing as M1₂ did.

• The reason of differences in ME coding:

  As per the DCM™ rules [N] is allocated no ME value, unless the sleeping/dozing is with some signs of discomfort or restlessness. Therefore, the number of M1₂’s time frames relevant for IRR evaluation became smaller as that of [N] became larger, resulting in the ME distribution of only 2[+1] and 1[+3] (Table 5), where ME = [+1] denotes “Neutral. No signs of positive or negative mood/ Brief or intermittent engagement,” and ME = [+3] denotes “Considerable positive mood/engagement.” Contrary to M1₂ having recorded many [N]s, M2₂ recorded many [D, +1]s instead, resulting in 11[+1]. M3₂ recorded [C, −1]s and [N]s in halves corresponding to 9[N]s of M1₂, resulting in as many as 4[−1] and just 3[+1].

As seen above, the differences in BCC coding for P2 was apparently caused by the difference in vision of each subject. Furthermore, the slight body motion of P2 could have been so subtle to have caused differences in interpretation by each subject. In other words, most of the time frames of P2 could have been recorded as [N]s if a mapper of less experience had mapped, while the 3 mappers ended up with different BCC coding with each other because of their high-level observation ability sensible of slightest body motion and/or very subtle expression of the participant. It became also clear that the differences of BCC coding have led to the ME coding differences, ended up with even lower IRR concordances.

Now that we looked into the Prep Map data per subject, we can find a quite similar pattern of difference regarding BCC coding of [N], [D], [K], and [C] in Prime Map (Table 5), demonstrating each subject mapped quite consistently. In principle, you need to conduct an IRR map before a full evaluation map, with another or some other mappers until each pair could achieve at least 70% concordance for evaluation purposes and 80% for research purposes. ^2,20 Considering their consistency in mapping, if the 3 subjects could have conducted an IRR map properly the results of their mapping might have ended up with one of the following coding agreements:

1. Both M2₂ and M3₂ agreed to code [N] regarding the time frames for which M1₂ coded [N], while M2₂ coded [D] or [K] and M3₂ coded [D], [K], or [C].

2. Both M1₂ and M3₂ agreed to code [D] regarding the time frames for which M2₂ coded [D], while M1₂ coded [N] and M3₂ coded [N], [K], or [C].

Table 6 shows the resultant concordance rates, given the above agreements were reached. CR0 and CR regarding P2 reached 70% or above, under both conditions (1) and (2), ended up with the total concordances CR0 and CR regarding 5 participants reached 70% or above, as well. As a reference, we calculated the total concordances regarding 4 participants, excluding P2 as singular, all of the concordance rates CR reached 70% or above, as shown in Table 6; condition (3). Thus, it seems to be highly probable that each pair of the 3 subjects could have achieved at least 70% concordance in Prep Map and Prime Map if they had conducted an IRR map properly and that utilization of the support system did not affect the mapping results.

Table 6.

Concordance Rate With Accommodated P2 Data in Case 2 Mapping.

Mapper pair and map category		Condition (1)		Condition (2)		Condition (3)
		P2_N	Total_N	P2_D	Total_D	exc P2	Total_e
	M12-M22
Prep Map (1 hour)	Relevant time frames	24	120	24	120	0	96
	Concordance rate: CR0 (%)	79	70	79	70	–	68
Prime Map (5 hours )	Relevant time frames	106	568	106	568	0	462
	Concordance rate: CR (%)	84	72	70	70	–	70
	M12-M32
Prep Map (1 hour)	Relevant time frames	24	120	24	120	0	96
	Concordance rate: CR0 (%)	96	81	96	81	–	77
Prime Map (5 hours)	Relevant time frames	106	558	106	558	0	452
	Concordance rate: CR (%)	89	75	83	74	–	72
	M22-M32
Prep Map (1 hour)	Relevant time frames	24	120	24	120	0	96
	Concordance rate: CR0 (%)	79	78	79	78	–	78
Prime Map (5 hours)	Relevant time frames	112	580	112	580	0	468
	Concordance rate: CR (%)	89	77	76	75	–	75

Abbreviations: P2, Participant 2; P2_N, P2 data accommodated to N; Total_N, total concordance rate using P2_N; P2_D, P2 data accommodated to D; Total_D, total concordance rate using P2_D; exc P2, excluding P2; Total_e, total concordance rate excluding P2.

According to the results of foregoing discussion, it was confirmed that the support system could be utilized for evaluation purposes without any problems in case 2, as well. Furthermore, the fact that the differences in coding regarding P2 among the 3 subjects in Prep Map appeared with a quite similar pattern in Prime Map indicates that the consistency of the subjects’ mapping policy was maintained even using the support system. Hence, the reliability of the support system was confirmed to a certain extent.

Usefulness of the Support System

We developed the DCM™ support system aiming at burden reduction and easier operation of DCM™ for starter-level mappers in particular and assessed its usefulness previously. ¹⁸ The obtained easiness level (E) by the improved support system was also all positive, as shown in Figure 6, which confirmed that the system was useful for high-level mappers, as well. However, the level (E) for item 14 in Table 2 was very low and many of the subjects expressed their dissatisfaction by taking notes using the system. No trouble was reported regarding the entry-form selection for 5 participants by the changes of GUI (Figures 2 and 4A). However, the disadvantage of the system regarding notes taking seemed to have become more critical by the separation of entry-forms for codes (Figure 4A) and for notes (Figure 4B). Further improvements, such as a seamless transition from coding entry form to notes entry form, are needed.

The other issue regarding the difficulties to “cope with 2 or more participants’ activities at the same time” or to “enter codes/notes timely for many participants” also became apparent in the actual mapping instead of the previous video mapping with just 2 participants with no interaction between the 2. However, we obtained a comment; such as, “It wouldn’t be a problem if I got used to operating the system, though. I may be able to cope with it using the common notes entry-form, which I noticed after the mapping,” as well. Therefore, we think we need a lot more feedback to come up with a plan to solve the issue.

One of the subjects, a basic user of DCM™ in case 3, who mapped 5 participants for the first time over 5 hours, gave us comments; “I could carry out the mapping without any problems, thanks to the support system. I don’t think I could have mapped 5 participants without it.” For a starter level mapper, it is very challenging to observe 5 participants at a time and quite tiring to continue such mapping for a long period. Thus, it was demonstrated that, with the help of the support system, a mapper can concentrate on observing/recording instead of having to pay attention to keeping track of time or to the complex DCM™ rules in actual mapping.

The feedback comments obtained in this assessment ended, as a whole, stating, “I need to get used to the system. I cannot make a firm comment on the usability of the system unless I use it several more times,” or “I do want to map using the system again after enough practice.” In fact, we only took some 30 minutes for instructions and practice of the support system before conducting the tests, therefore, the subjects were not explained all functions nor getting used to the operation of the system. In order to make improvements of the system based on usability assessment, it is therefore necessary to offer chances for many users to get used to the system and then obtain as much feedback as possible from them.

We received some other responses of open question regarding the usefulness of the support system as follows:

• I do not feel fatigued with the support system, indeed. I do wish the system would be put into practical use!

• Development of the system is challenging, but I look forward to its finalization.

• The support system can serve as assistance in education; for example, review of the rules, decision of coding, and so on.

• It would be nice if data were processed and graphs would come out right after the mapping.

It would be nice if I could compare coding with other mapper easily after mapping; for example, only the time frames with coding disagreements would be displayed automatically.

Regarding the last 2 needs, it is technically easy to realize. Furthermore, it is not technically difficult, for example, to project, in a meeting form, the mapping data on a screen in a tabula worksheet form for reviewing/correcting the data and make summary documents. As Griffiths et al ¹⁶ identified further barriers caused by the burdens of data analysis, paperwork, and report writing, there remains issues for the DCM™ support system to reduce mappers’ burden, which we plan to include in the future study.

Step Toward the Practical Use of the Support System

We conducted the practicality assessment of the support system with extreme care by selecting 2 typical cases where we could obtain highly reliable mapping data. Yet, it is necessary to conduct assessment with larger size samples in order to ensure the reliability of the assessment. It should be noted, however, that evaluation tests should be carried out under the following conditions for valid assessment:

• (1) Recruit subjects (users) who adequately have got used to operating the support system.

• (2) Obtain highly reliable data for evaluation.

The easiest way to satisfy condition (1) may be to select a limited number of subjects and let them have enough practice to use the system. It may cause a problem, however, that the candidates will be limited to high-level mappers in order to satisfy condition (2), which ends up with limited number of subjects. In addition, feedback for the system improvement will be limited due to the limited number of subjects, although it is necessary to obtain as much feedback as possible to improve and finalize the support system. Hence, the authors propose to let a wide range of mappers of all levels trying out the support system as is in their usual mapping activities, supposing that the practicality of the system is confirmed to a certain degree by this assessment. The aim of this proposal is to promote finalization of the DCM™ support system for practical use. There will be a big chance to conduct mapping of a larger sample size with a longer period of time for proper evaluation of the support system, once skilled users of the system have grown and the system has been highly improved. It may become possible, at that stage, to evaluate the reliability and usefulness of the support system not only for evaluation purposes but also for research purposes. The system can be used for research purposes to evaluate, for example, the validity of 5-minute time frame. Formerly, Thornton et al ²¹ assessed whether the 5-minute time frame was able to provide a meaningful representation of actual events. They compared the data obtained by 2 different methods, DCM™ mapping and continuous-time sampling, to discuss the issue. The support system makes it possible to analyze the identical raw DCM™ data under different time frame lengths because the mapping data obtained using the support system is practically continuous, and therefore, it can be analyzed with an arbitrary length of time frame.

As described above, the advantage of adopting ICT to DCM™ is to help increase mapping accuracy and reduce mappers’ burden not only during mapping but also after mapping regarding data processing and paperwork such as making documents for feedback by reducing the time required. In addition, it may be possible to analyze and develop the DCM™ method from research aspect. The DCM™ support system is still halfway for practical use officially; however, we think it is important to let many users try out the system as soon as possible to accumulate users’ experiences and improve the system in order to pursue foregoing possibilities.

Limitations

Some of the limitations of this study are (1) the small sample size and (2) the seating arrangement of the 3 subjects (mappers) side by side in a row, which made them difficult to observe a specific participant equally precisely in case 2.

We selected the subjects very carefully in this study in order to obtain highly reliable mapping data for the assessment, which ended up with conducting the tests in a small sample size. However, we selected 2 typical cases in which highly reliable data were assured, which made the tests probably more general than just a couple of case studies. In order to conduct assessment in a larger sample size, it is necessary to grow the number of users of the support system first to prepare for it, as described previously.

Regarding the limitation (2), we asked each team of 3 subjects (mappers) to be seated closer together so that they have the same view of the participants, expecting to obtain good data for a fair comparison. However, it did not work right regarding a specific participant in case 2. We learned from the results that mapping could be carried out more properly if we had let each mapper sit or move around as they want so that they could best observe the participants and make detailed notes as usual. Moreover, it is more desirable to evaluate the support system using data obtained from mappers who made the best use of DCM™ in such a way.

Conclusions

Dementia Care Mapping is an observational method that has been widely used worldwide to implement person-centered care in dementia care settings for over 20 years. However, recent researches have reported mixed results on the efficacy of DCM™, and relevant studies identified barriers and facilitators to DCM™ implementation, pointing out that good understanding and strong support by managers and staff were quite important for successful implementation of DCM™. Also highlighted by the studies were the complex nature of DCM™ and high requirement for skills and qualities of the mapper role.

The authors have developed an ICT-based DCM™ support system in order to solve the complexity of the method and reduce mappers’ burden and assessed its usefulness using videos previously. As a result, we obtained users’ positive ratings of the system for burden reduction and its usability. In this article, we assessed practicality of the improved support system through actual mapping in dementia care facilities. As the results, the concordance rates in Prime Map reached higher than 70%, thus it was confirmed that the support system could be utilized for evaluation purposes without any problems.

For the next step, we need to conduct an evaluation of the support system with larger size samples over a longer period of time to ensure the reliability of the assessment. Prior to that, however, it is necessary to grow the number of users who get used to operating the support system and to make further improvements based on their feedback; we can conduct highly reliable assessment only after this. The result of this study demonstrated that the improved DCM™ support system has reached the level for trial use in actual mapping for further evaluation.