An Evaluation of the Usability of a Computerized Decision Support System for Nursing Homes

Abstract

Background

Computerized decision support systems (CDSSs) have the potential to significantly improve the quality of nursing care of older people by enhancing the decision making of nursing personnel. Despite this potential, health care organizations have been slow to incorporate CDSSs into nursing home practices.

Objective

This study describes facilitators and barriers that impact the ability of nursing personnel to effectively use a clinical CDSS for planning and treating pressure ulcers (PUs) and malnutrition and for following the suggested risk assessment guidelines for the care of nursing home residents.

Methods

We employed a qualitative descriptive design using varied methods, including structured group interviews, cognitive walkthrough observations and a graphical user interface (GUI) usability evaluation. Group interviews were conducted with 25 nursing personnel from four nursing homes in southern Norway. Five nursing personnel participated in cognitive walkthrough observations and the GUI usability evaluation. Text transcripts were analyzed using qualitative content analysis.

Results

Group interview participants reported that ease of use, usefulness and a supportive work environment were key facilitators of CDSS use. The barriers identified were lack of training, resistance to using computers and limited integration of the CDSS with the facility’s electronic health record (EHR) system. Key findings from the usability evaluation also identified the difficulty of using the CDSS within the EHR and the poorly designed GUI integration as barriers.

Conclusion

Overall, we found disconnect between two types of nursing personnel. Those who were comfortable with computer technology reported positive feedback about the CDSS, while others expressed resistance to using the CDSS for various reasons. This study revealed that organizations must invest more resources in educating nursing personnel on the seriousness of PUs and poor nutrition in the elderly, providing specialized CDSS training and ensuring that nursing personnel have time in the workday to use the CDSS.

1. Introduction

Health care is an information-intensive practice. Providing high-quality care depends heavily on the ability of nursing personnel to identify, access, interpret and integrate relevant data and information in their clinical decision making. Nevertheless, adoption of Information and Communication Technology (ICT) has been slow [1]. One example of such technology is clinical computerized decision-support systems (CDSSs), which have been suggested to improve the clinical decisions of nursing personnel with the goal of improving health care safety, quality and efficiency; reducing costs and improving patient outcomes [2–4]. Clinical CDSSs can be defined as specialized information systems that are purposely designed for ‘end users’ decision making in the health care setting to provide relevant information that can be integrated with the characteristics of individual patients at the point of care. The clinical CDSS output (i.e., results) are intended to provide nursing personnel with recommendations for action [5]. Studies of CDSSs in the nursing home setting are uncommon, although their potential positive impacts are recognized [6, 7].

The purpose of the clinical CDSS evaluated herein was to support the detection and prevention of serious and common problems among nursing home residents [8]. It was integrated into the electronic health record (EHR) system called Prosys (www.alvara.no). The CDSS was developed from two research-based risk assessment instruments: the Risk Assessment Pressure Scale (RAPS) for pressure ulcer (PU) risk screening [9] and the Mini Nutritional Assessment (MNA^®) tool [10] for nutritional status screening. Based on the results of these assessments, the CDSS presents evidence-based interventions to support nursing care planning that were developed from clinical experience, previous research and systematic reviews on PUs and malnutrition [11, 12]. The CDSS was developed by various stakeholders (e.g., an information system developer, a software vendor, nursing personnel and health care researchers). Validation and testing of the CDSS was conducted by a group of ‘super-users’ who had expertise using the EHR and had also been actively involved in the development process. ►Figure 1 shows an example screen of the CDSS.

Figure 1.

Screenshot of the computerized decision support system, showing a part of the Risk Assessment Pressure Scale (RAPS), a research-based risk assessment screening instrument. On the left, different risk assessment scales can be chosen that can then be completed with residents’ data. On the right, guiding text is available to allow the user to select interventions based on RAPS score. The Prosys graphical user interface © copyright Alvara AS

1.1 Review of the Literature

Clinical CDSSs range from passive systems that assist nursing personnel with guidelines and research evidence [5] to more active systems that integrate data from multiple sources and provide patient-specific recommendations to improve nursing or medical diagnoses, treatment decisions, drug administration or preventive interventions [2, 5]. Several systematic reviews of studies of clinical CDSSs provide examples of systems that are available for health care personnel in different settings; however, there is a paucity of studies that evaluate these systems [3, 13].

A review of eight studies showed that the CDSSs used by nursing personnel were mostly adopted without prior evaluation of the clinical content on which the CDSSs were based [3]. Another metaanalysis based on 17 papers found that evidence-based CDSSs may support the implementation of guidelines-based care and improve patient outcomes, although barriers such as a lack of administrative support, the time required to learn and implement the new technology, and deficiencies in the EHR systems were reported [13]. In addition, contextual aspects such as social and organizational factors are important to include in an evaluation to enhance the understanding of the function and effectiveness of the CDSS [14]. Also, it has been reported that CDSSs would be used more if physicians had more time available [15].

A central feature of the success of CDSSs in health care is designing the graphical user interface (GUI) to provide effective guidance for nursing personnel [4]. The GUI is the part of the computer application that allows users to interact with electronic devices [16]. One common usability evaluation technique is the cognitive walkthrough observation [17]. In a cognitive walkthrough, a user performs tasks while a usability expert evaluates the system design. The usability expert observes errors made by the user, listens to verbal feedback from the user and notes problems in completing tasks in a timely manner [17]. The number of usability evaluations conducted on ICT in laboratories or simulated settings is increasing, but studies performed in natural settings are still rare [18].

1.2 Theoretical Framework

Several theoretical models have been presented in the information systems literature to explain the factors that influence ICT implementation. DeLone and McLean’s model of information system success [19] and the Technology Acceptance Model (TAM) [20] have been used to explain the factors that are most likely to predict positive attitudes and increase the likelihood of adoption of new technology. The Task Technology Fit (TTF) [21] model addresses utilization from a different perspective and focuses on the concepts that are most likely to predict performance impact.

In the TTF model, technology characteristics are defined as the underlying features of an information system, and task characteristics are defined broadly as the actions taken by individuals. The dependent variable of task characteristics and technology characteristics is the task-technology fit. Task-technology fit may be defined as “when a technology provides features that ’fit‘ the requirements of a task” [22, p.214], and it is measured by how well the system functionality corresponds with the task needs. TTF predicts (or can be measured by) actual utilization and performance impacts. If the technology provides features corresponding to the task, then it is presumed to have a positive impact on the performance [22]. Eight dimensions have been identified that influence TTF: data quality, the ability to locate data, authorization to access data, data compatibility between systems, production timeliness, system reliability, training and ease of use, and the relationships between information system (IS) developers and users [22, 23]. In this study we are measuring the fit as a part of facilitators and barriers through group interviews and two usability evaluations (e.g., cognitive walkthrough observations and a usability evaluation questionnaire).

Health care researchers argue that the dimension of fit must be evaluated when introducing ICTs in health care settings [24, 25]. In this study, we have applied the TTF model to determine whether the CDSS fits the needs of nursing personnel in nursing homes.

The importance of the congruence between the selected technology and the task to be accomplished is often overlooked in the development and implementation of health care information systems [26], and only a few studies have used the TTF model to study whether ICT is utilized effectively [25, 27]. However, some studies have shown that the TTF model provides a solid theoretical framework for investigating, evaluating and explaining the performance impact of ICTs in health care settings [25, 28]. Some have argued that a high level of TTF will lead to the more effective and efficient use of ICT and will engender positive performance impact [21]. Cane and McCarthy’s [27] metaanalysis supports the use of the TTF model rather than TAM alone because the TTF model focuses on utilization rather than intention to use technology. Ward and co-workers [29] examined the attitudes of health care personnel toward ICT and identified issues such as system flexibility and usability, appropriate education, training, and the users’ confidence and experience in using an ICT as important factors affecting the utilization of ICTs.

Health care organizations spend billions of dollars on ICT investments [1]. Given the large expense of these systems and their importance to patient outcomes [30], it is critical to evaluate the usability of ICTs to ensure that they are achieving their intended outcomes.

2. Objective

This study describes the facilitators and barriers that impact the ability of nursing personnel to effectively use a clinical CDSS both for planning and treating PUs and malnutrition and for following the suggested risk assessment guidelines for the care of nursing home residents.

3. Methods

3.1 Setting

The most recent data available reports that Norway has 998 nursing homes that care for 41,052 residents [31]. This is a significant sector of Norway’s 5 million people. Almost all municipalities in Norway have implemented the EHR [32]. According to the Health Care Personnel Act [33] in Norway from 2001, all personnel who are responsible for the examination, diagnosis, treatment and follow-up of nursing home residents are obliged to document resident care plans in the EHR. The nursing home personnel from four nursing homes, who participated in this study, were the only ones using the CDSS. They had used the EHR for several years, but participants’ level of experience with using the CDSS for nursing care planning varied.

3.2 Design and Sample

This qualitative descriptive study consisted of group interviews and two usability evaluations: cognitive walkthrough observations and a usability evaluation questionnaire.

3.2.1 Sample for Group Interviews

A sample of 25 nursing personnel, composed of Registered Nurses (RNs, n = 19), a Special Needs Educator (n = 1), and Nurse Aides (NAs, n = 5) participated in four group interviews. All participants were female, and age ranged from 24 to 65 years old (mean = 39.5 years, SD = 11.9). As shown in ►Table 1, six participants had education in older people care (e.g., psychiatric nursing, health informatics or management), 21 worked both day and evening shifts and participants reported a range of employment levels (i.e., from 42% to 100% of full-time employment). Of the 21 participants who reported working day and evening shifts, nine worked full-time. All participants had approximately eight months of experience using the CDSS.

Table 1.

Characteristics of group interview participants (n = 25)

Characteristics
	n	%
Gender, female	25	100%
	years	Mean	SD range
Age (years), mean, SD (range)	39.5	11.9	24–65
	n	%
Nursing personnel
• Registered Nurses	19	76
• Special Needs Educator	1	4
• Nursing Aides	5	20
Participants with specialist education in older people care, psychiatric nursing, health informatics or management	6	24
Working day and night shift, n (%)	21	84
	%	Mean	SD range
Percent of full-time duty, mean	84.7	15.7	42–100

3.2.2 Sample for the Usability Evaluation

Two RNs and three NAs from three different units in one nursing home participated in the cognitive walkthrough observations and completed the usability evaluation questionnaire. All worked full time; two were less than 30 years old, one was between 30 and 39 years old and two were between 40 and 49 years old. All five reported that they were experienced with computers. Three participants described themselves as experienced users and two as inexperienced users of the CDSS. In many usability studies it has been shown that a sample size between 5 and 8 participants is a sufficient number and will identify 80% of the problems and increasing the number of participants does not significantly increase the percentage of problems found [34].

3.3 Interview Guide

A semi-structured interview guide for the group interviews was developed based on previous research [35–37]. The following areas were explored during the interviews:

• 1.training on how to use the CDSS,
• 2.use of the CDSS in the unit,
• 3.perceived facilitators and barriers, and
• 4.experiences gained from using the CDSS.

3.4 Scenarios and Questionnaire

The scenarios in the usability evaluation described three different patient datasets that allowed participants to do PU risk assessments. The usability evaluation questionnaire was based on an instrument adapted for a health care environment [38] (►Table 2). The questions were translated into Norwegian, and the instrument was pretested by two RNs to verify that it was understandable.

Table 2.

CDSS usability response scores from “super-users” (n = 5)

Question	Median
4. The system was easy to use.	7.0*
8. The font size was appropriate.	7.0
9. The font style was appropriate.	7.0
10. The labels that described the functions made sense to me.	7.0
11. I was able to find functionality where I expected.	7.0
14. The amount of information was appropriate (i.e., not overwhelming or too sparse).	7.0
15. I could become productive quickly using the main menu in the system.	7.0
19. The system was easy to learn to use.	7.0
1. The functions of the system met my needs.	6.0
3. The available functionality of the system was complete.	6.0
5. The system was enjoyable to use.	6.0
7. I was able to navigate easily while using the system.	6.0
12. The organization of functions made sense to me.	6.0
16. I was able to develop a care plan using the system.	6.0
17. Interventions presented in the system were relevant.	6.0
18. I have got enough education to use the system.	6.0
21. Overall, I am satisfied with the system’s capabilities.	6.0
2. The function of the system worked as I expected.	5.0
6. The system was flexible.	5.0
13. I was able to find functionality where I expected.	5.0
20. It took too much time to learn to use the system in relation to the benefits of using the system.	1.0**

^* (Disagree 1 – Agree 7)

^** (Disagree 7 – Agree 1)

3.5 Procedure

3.5.1 Group Interviews

The administrators of the nursing home units were asked to recruit nursing personnel to voluntarily participate in the group interviews. The interviews were conducted at the work site, with four to seven participants from the same nursing home in each group. The interviews lasted between 60 and 90 minutes and were tape recorded and transcribed verbatim.

3.5.2 Usability Evaluation

For the cognitive walkthrough observation, each nursing personnel was asked to independently complete three tasks based on patient care scenarios. The evaluation was done at the participant’s workplace and lasted between 45 and 60 minutes. As each participant completed the task, she was asked to “talk out loud” and express her thinking while using the CDSS. The researcher observed the participant and made notes about comments, questions, frustrations, and the method used by the participant to complete the task [17], as shown in ►Table 5.

Table 5.

Results from the cognitive walk-through (n = 5)

Task	Participant 1	Participant 2	Participant 3	Participant 4	Participant 5
Task 1
Time (min)	3:59	3:58	2:05	3:53	3:31
Comments	The texts on the labels are instructive.	Used the guiding text.			Used the guiding text all the time.
Frustration comments		Starting the assessment could be confusing.	Differences between the guiding texts to the same navigation.	Must move windows to read texts in the user interface.
Navigation of system		Good with red text on the labels.
Task 2
Time (min)	5:26	3:20	4:18	4:50	5:56
Comments	The guiding texts are important. The dropdown menus are important.	The guiding texts give good instructions.	Too much space between the guiding texts. All information should be possible to see in one window.	Using the guiding texts active.	Saving should have been done automatically.
Frustration comments	The item “sensory perception” in the guideline is difficult to score. Bothersome to change windows in the system from assessment to choose interventions.	If the guiding texts gave more instructions, the dropdown menus would have been unnecessary.	Two difficult items in the assessment: physical activity and mobility.	Must move windows to read texts in the user interface.	Input data must be saved after every input.
Navigation of system	Four click for every score in the guidelines.		Four clicks for every score in the guidelines.	Changed between using the mouse and the keyboard for the same navigation.
Task 3
Time (min)	10:33	10:32	4:25	3:48	4:03
Comments	Should have been easier to find the right labels, suggested more guiding text.	When printing the care plans, dates have to be typed every time manually.
Frustration comments	There are too many windows and labels up on the screen at the same time.	Must type F4 to see the guiding text on serum albumin.
Navigation of system	Good with red text on the labels.	The guiding texts are useful.			The guiding texts are useful.

After the cognitive walkthrough, each participant completed a usability evaluation questionnaire that elicited her demographic information, educational background and experiences with using the CDSS. The first author conducted both the cognitive walkthroughs and the group interviews.

3.6 Ethical Consideration

Approval for this study was obtained from the Regional Committee for Medical Research Ethics in southern Norway (REK Sør, reference number S-07212b) and from the Norwegian Social Science Data Services (project number 16822). The participants received oral and written information about the study and the voluntary nature of their participation, and all signed a written informed consent. All participants were assured confidentiality.

3.7 Data Analyses

The written transcripts were processed in a word-document with three columns for each unit from the group interviews and the data from the cognitive walkthroughs were analyzed using manifest content analysis, using the visible and clear components of the text, without interpreting the meaning [39]. All interviews were listened to in their entirety, and the transcribed data were read thoroughly to gain an overview and a general impression of the data. The transcripts were then analyzed to identify meaning units or coding units, condensed meaning units and codes related to the study objective [39]. Two researchers independently coded each of the interviews, and when differences in coding occurred, the researchers discussed each coding discrepancy until an agreement was reached.

4. Results

4.1 Facilitators Experienced from Implementing and Using the CDSS

The facilitator results were categorized into two meaning units: professional and software design facilitators.

4.1.1 Professional Facilitators

The professional meaning unit included several important insights. Participants described that as more emphasis was placed on PUs, nutrition, and documentation, they had correspondingly more motivation to use the CDSS and therefore increased the number of interventions in these areas of patient care. They also expressed that the CDSS had facilitated the identification of residents in need of nutritional interventions, which also increased the usage of the CDSS. As one participant described, “After the implementation of the CDSS, we are now more focused on the kind of food we order for the residents”, and “When screening a new resident, I can see from using the CDSS the new interventions that are necessary, what we can work on and what can wait”.

4.1.2 Software Design Facilitators

Some participants in every group interview expressed that the CDSS was easy to use. For example, after the participant entered the resident’s data, the system suggested different interventions that she could select.

In addition, many participants stated that the availability of ‘super-users’ and support from unit managers were important facilitators in the implementation process. The participants recommended lectures and individual CDSS training as supporting strategies when implementing the CDSS. ►Table 3 presents the categories of facilitating factors.

Table 3.

Facilitating factors during implementation and use of the CDSS in four nursing homes, based on group interviews with nursing personnel (n = 25)

Professional	Design-software
Emphasis on nutrition and pressure ulcer prevention	Ease of use
Focus on interventions	Usefulness
Attention on improving documentation
Support from super-users
Support from the nearest manager

4.2 Barriers Experienced in Implementing and Using the CDSS

The barriers results were categorized into four meaning units: organizational, individual, task-fit and design-software, as presented in ►Table 4.

Table 4.

Perceived barriers during implementation and use of the CDSS in nursing homes, based on group interviews with 25 nursing personnel

Organizational	Individual	Task-Fit	Design/software
Lack of training Lack of information for nursing personnel Challenging to organize the use into the daily workflow Lack of equipment to measure body weight Lack of work stations Lack of routines for risk assessment personnel Too large a workload Preference for oral routines for inter-shift reports	Lack of involvement in the implementation of the CDSS Lack of computer skills for using the CDSS Resistance to using computers Feel no need for care plans Lack of basic knowledge in prevention of malnutrition and PUs among nursing	Feels unnecessary to use guidelines	The nutrition plan was not fully integrated into the EHR Lack of good user interface in the EHR system The CDSS has lack of logic when navigating to choose interventions The EHR system was difficult to use

4.2.1 Organizational Barriers

The participants described the organizational barriers as lack of training, lack of information about the implementation of the CDSS, lack of equipment to measure body weight and arm and leg circumference, lack of computer work stations, lack of clinical knowledge among personnel and lack of routines for the systematic risk assessment of residents in the unit.

Participants in the groups identified lack of training as a barrier. They explained that all personnel in the unit should have had an opportunity to participate in the training, and furthermore, it should not have been optional. The next organizational barrier, lack of information, was expressed as follows: “the information (e.g., updates) about the CDSS implementation was good in the beginning, but then a lot of organizational changes occurred, and this made it more difficult for everyone to be kept informed”.

To complete the resident assessments in the CDSS, all the nursing homes needed equipment to measure body weight and arm and leg circumference. The lack of this specialized equipment was reported as a barrier to CDSS use. Another organizational barrier was the lack of available computer work stations. Participants stated: “We should have more work stations for documentation in all the units. Then we would not have to wait to gain access to a work station”.

In all of the interviews, a lack of clinical knowledge among personnel and a lack of routines were reported. The following statements by participants supported these conclusions: “One of the problems that limits the use of the CDSS is that we have so many personnel without formal education. Some of the less educated personnel think the residents eat well enough, and nutrition interventions are not needed”. Another organizational barrier identified was the paper-based documentation in some parts of the EHRs; participants expressed concerns about routines for updating the information that was printed on paper and stored in the unit rather than in the EHR. Therefore, the data in the CDSS were incomplete, and the CDSS was less effective.

4.2.2 Individual Barriers

The individual barriers included lack of participation in the CDSS implementation process, lack of computer skills, preferences for oral information exchanges, a lack of motivation to use care plans, and resistance to using computers. The resistance to using computers was also discussed with respect to the older age of many nursing personnel; some participants perceived that it was easier for younger nursing personnel to use computers as one participant explained “It becomes a problem for nursing personnel, because lots of nursing personnel have a high age and of course they are more reluctant”.

4.2.3 Task-Fit Barrier

The lack of task-fit of the CDSS was discussed, and some participants felt there was no need to screen the residents for nutritional risks because they believed that none of the residents in their unit were undernourished. However, some participants expressed surprise when they became aware of the Body Mass Index (BMI) results (this is one of the CDSS capabilities): “It is difficult to just look at the residents and assess their nutritional status. (The residents may have a lack of muscle mass that cannot be recognized by the human eye)”. These comments were most often made by personnel who were less comfortable using the computer or who did not follow this protocol on a routine basis, and was mentioned in all four group interviews.

4.2.4 Design and Software Barriers

Design and software barriers were categorized as EHR user interface challenges due to the poor integration of the CDSS with the EHR. The participants stated that the EHR/CDSS graphical user interface could be more logically designed (e.g., the integration of various applications on mobile phones). They wanted additional functionality, such as standardized care plans and a reminder function for continually updating care plans.

4.3 Usability Evaluation Questionnaire

The items in the usability questionnaire all received high scores from the participants regarding their satisfaction with the system, which was somewhat surprising because the results did not align with the feedback from the group interviews. This incongruity could exist because the participants in the cognitive walkthrough also answered the questionnaire, and they were more familiar with the systems than some of the participants in the group interviews. They were also positive regarding the introductory training and support in using the CDSS, as displayed in ►Table 4. Questions 2, 6 and 13 with the lowest scores (i.e., Md 5) focused on the functions of the system, the ability of the system to find capabilities where expected and the flexibility of the system.

When asked what they liked about the system, all five participants reported ease of use. Drawbacks of the system were that the CDSS was not integrated well enough with the EHR and that the CDSS application was difficult to find while browsing in the EHR system. Three key improvements were suggested: improve the display design of the interventions capability, increase the visibility of all the details under the main intervention headings so that the different intervention choices are clear (►Table 2) and make it easier to save data when performing a risk assessment. The participants stated that despite these shortcomings, the CDSS was a great tool. They especially liked the link between the CDSS and the care plan. This feature provided valuable information for daily care planning.

The TTF performance impact was measured by how well the nursing personnel accomplished the three scenarios with the RAPS [9]. The participants showed minimal variation in their rating of risks and successfully completed the scenarios. However, the time needed to complete the three cases varied between 02:05–03:59 minutes for task 1, 03:20–05:56 minutes for task 2 and 03:48–10:33 for task 3 (►Table 5).

The cognitive walkthrough identified four major barriers. First, as also identified above, the CDSS was not integrated well enough into the EHR system. The users had to spend too much time navigating in the EHR system to locate the CDSS application. The participants stated that navigation in the system would have been easier if the detailed interventions under the headings (e.g., PU prevention) were more obvious (►Fig. 1) and that the users should have had an opportunity to see all the text under each heading (e.g., regular repositioning, providing pressure relieving mattresses and high protein nutrition, etc.). Second, the participants noted that data in the EHR had to be saved after every input, which required three clicks for every rating; otherwise, the data would be lost, making the system unnecessarily cumbersome and frustrating for the user. Third, the users had to move windows on the screen to view all the guiding text. Finally, the users wished for better graphic visualizations (e.g., charts) in the CDSS to provide a better assessment of the patient (e.g., present the changes in body weight in a numeric graph).

5. Discussion

Nursing personnel expressed both positive and negative reviews of the CDSS. Many perceived that the CDSS contributed to more professional care by those personnel who had more nursing domain-specific knowledge on the prevention of PUs and malnutrition. After using the CDSS, they reported that the number of relevant interventions increased, and they perceived there was more focus on documentation. They also described the CDSS as easy to use.

However, the participants emphasized that the CDSS could have been better integrated into the EHR and that the system lacked flexibility and logical flow. Participants identified the following organizational factors as barriers to CDSS implementation: lack of training, lack of information about the CDSS implementation, high workload, lack of computer skills and lack of clinical knowledge. Individual barriers such as lack of involvement in the implementation, lack of skills required to use the CDSS and resistance to use computers were also concerns.

Barriers such as time constraints in everyday work, lack of access to work stations, and lack of computer skills have also been reported in previous studies on the use of a guideline-based CDSS by general practitioners [40, 41]. Lack of adequate training was reported as an important barrier to successful implementation of the CDSS in four case studies from hospitals in England [14]. Strategies for training and education are important to consider in the early stages of planning for the implementation of new technologies in nursing homes. Overall, in our study, nursing personnel perceived the CDSS as easy to use, but at the same time, there was a lack of knowledge, skills and motivation to use the new technology.

5.1 TTF Model Implications and Discussion

Due to the qualitative nature of this study, our findings can be related to the eight dimensions of the TTF model: locatability of data, data quality, authorization to access data, data compatibility between systems, production timeliness, system reliability, training, ease of use/training, and the relationship between the information system (IS) developer and the users [22].

The nursing personnel expressed concerns about the data quality when paper and electronic data was used concurrently. The locatability of data can be evaluated by the ease with which the user can find (“locate”) and identify the available data [22]. Based on the experiences of the nursing personnel in our sample, the CDSS allowed them to focus more on the prevention of PUs and malnutrition, relevant interventions and proper documentation. Their descriptions indicated that the CDSS provided valuable data for nursing care.

The data quality dimension is defined as the use of the correct data with an appropriate level of detail. In two nursing homes, nursing personnel documented the risk assessments on paper rather than using the CDSS, and then an assigned nurse transferred the data to the EHR. These nursing homes had a parallel paper system with care plans printed from the EHR. Another Norwegian study showed that it was a challenge for nursing personnel to update data when paper-based documentation was used in parallel with a computerized system [42]. Such parallel documentation may have influenced data quality and the availability and accessibility of data and thus, may have had a negative impact on the participants’ perception of the usefulness of the system and the fit between technology and task.

Challenges due to authorization to access data were not mentioned in the group interviews, probably because all of the nursing personnel who participated had access to the EHR. Because only one system and one database was tested, data compatibility between systems was not relevant. Likewise, production timeliness (i.e., where the Information Systems groups meets a predefined production schedule) did not apply in this case. Because the CDSS was part of the overall EHR system, system reliability was not an issue.

The ease of use and training dimensions addressed the ease with which the users could accomplish what they wanted to do and their access to training, respectively. As mentioned previously, the CDSS was easy to use. However, training was inconsistent. The lack of control over who participated in the training may have influenced the results, and although the nursing personnel received training, they did not necessarily learn to manage the system.

The relationship between the Information System developer and the users occurred during the CDSS development process through regular meetings to discuss requirements, analysis and design. Only the nursing personnel who were part of the system development collaboration team had a relationship with the system developers.

Another study reported that challenges with the GUI and browsing in the EHR might have influenced the data quality, data locatability and data compatibility in the EHR, which may have then negatively influenced the nursing personnel using the CDSS. Data quality, data locatability and data compatibility were shown to influence TTF in a survey of registered nurses in hospital settings [25]. Training needs and ease of use have also been reported to influence TTF by registered nurses in hospital settings [25].

In summary, the three most relevant TTF dimensions were data locatability, data quality and ease of use/training. Overall, it appears that the CDSS technology does reasonably “fit” the task, and the job performance of the users who were knowledgeable about PU and nutrition improved.

5.2 Reflection from the Usability User Interface Evaluation

A clinical CDSS must be well designed and properly used to increase the quality of care for nursing home residents and to reduce health care costs [4]. Findings from this explorative, qualitative study support the need to focus on data quality, data locatability, data compatibility between systems, system reliability, training and ease of use, and all the other aspects identified in previous research on the dimensions that may influence TTF [25, 28] in nursing homes. Nursing domain-specific knowledge needs to be included as an important feature for the performance of nursing personnel, and it has not been covered by the TTF model. Potential users must be convinced that the technology is helpful in their caring for residents before they will adopt it. To convince the nursing personnel, the training must be modified; demonstrating the usefulness of the CDSS may be required rather than simply teaching nursing personnel how to use the system. The organization must ensure that the users understand the usefulness of the system so that they are motivated to use the CDSS.

There have been only a few usability evaluations performed on CDSSs used by nursing personnel [43]. Insights from the usability evaluation questionnaire and the cognitive walkthrough observations provide useful recommendations to guide the further development of CDSSs for nursing home care. In addition, increased and required training can also improve both user satisfaction and user performance.

5.3 Methodological Considerations and Limitations

These findings are consistent with those of earlier studies in other health care settings that documented the importance of user involvement to increase EHR usage [37, 44]. A contribution of this study is that it shows that some dimensions of the TTF model can be used as a framework to better understand the dimensions that determine fit between the task and technology in the health care context. However, we suggest further theory development of the TTF model to include domain-specific knowledge as one of the dimensions that predicts performance.

The first author, who conducted the group interviews, the usability evaluation and the cognitive walkthrough observations, had been involved in the implementation of the CDSS, which may have made the study participants reluctant to express critical opinions of the CDSS. Alternatively, it may have made them more at ease to freely express their opinions.

There was no control over how extensively the participants had actually used the CDSS in the intervention study, and although administrators were asked to recruit personnel with considerable experience, two of the participants described themselves as novice users. This limitation was due to the difficulty of finding nursing personnel available to participate. Another limitation was that all the participants in the usability evaluation and the cognitive walkthrough were recruited from only one of the four nursing homes involved in the intervention, which may limit the transferability of the findings [39]. Despite these shortcomings, a major strength of this study is the rich data generated from a natural setting, which provided valuable insights into the experiences of nursing personnel using a CDSS.

6. Conclusions

Overall, we found a disconnect between two types of nursing personnel. Those who were comfortable with computer technology reported positive feedback about the CDSS, while others expressed resistance to using the CDSS for various reasons. The TTF model can be used to understand some dimensions of fit between task and technology. This study revealed that organizations must invest more in educating nursing personnel on the seriousness of PUs and poor nutrition in the elderly, providing specialized CDSS training and ensuring adequate time in the workday to allow for utilization of the CDSS.

Implications of Results for Practitioners and/or Consumers

The findings from this study can help plan and accomplish other CDSS implementation projects in nursing homes. To ensure successful implementation of CDSSs, it is crucial both to create a supportive environment with information and training about the CDSS and the targeted nursing knowledge and to carefully integrate the CDSS into the EHR so that it fits with and supports the nursing workflow. We recommend that further research on the implementation of CDSSs in health care should be supplemented with usability evaluations.

Conflict of Interest

The authors declare no conflicts of interest in the research.