Skip to main content
PMC home page
Implementation Science : IS logoLink to Implementation Science : IS
. 2012 Mar 2;7:13. doi: 10.1186/1748-5908-7-13

The effect of a complementary e-learning course on implementation of a quality improvement project regarding care for elderly patients: a stepped wedge trial

Lotte Van de Steeg 1,✉, Maaike Langelaan 1, Roelie Ijkema 1, Cordula Wagner 1,2
PMCID: PMC3310782  PMID: 22385554

Abstract

Background

Delirium occurs frequently in elderly hospitalised patients and is associated with higher mortality, increased length of hospital stay, functional decline, and admission to long-term care. Healthcare professionals frequently do not recognise delirium, indicating that education can play an important role in improving delirium care for hospitalised elderly. Previous studies have indicated that e-learning can provide an effective way of educating healthcare professionals and improving quality of care, though results are inconsistent.

Methods and design

This stepped wedge cluster randomised trial will assess the effects of a complementary delirium e-learning course on the implementation of quality improvement initiative, which aims to enhance the recognition and management of delirium in elderly patients. The trial will be conducted in 18 Dutch hospitals and last 11 months. Measurements will be taken in all participating wards using monthly record reviews, in order to monitor delivered care. These measurements will include the percentage of elderly patients who were screened for the risk of developing delirium, use of the Delirium Observation Screening scale, use of nursing or medical interventions, and the percentage of elderly patients who were diagnosed with delirium. Data regarding the e-learning course will be gathered as well. These data will include user characteristics, information regarding use of the course, delirium knowledge before and after using the course, and the attitude and intentions of nurses concerning delirium care.

Setting

The study will be conducted in internal medicine and surgical wards of eighteen hospitals that are at the beginning stages of implementing the Frail Elderly Project in the Netherlands.

Discussion

Better recognition of elderly patients at risk for delirium and subsequent care is expected from the introduction of an e-learning course for nurses that is complementary to an existing quality improvement project. This trial has the potential to demonstrate that e-learning can be a vital part of the implementation process, especially for quality improvement projects aimed at complex health issues such as delirium. The study will contribute to a growing body of knowledge concerning e-learning and the effects it can have on knowledge as well as delivered care.

Trial registration

Netherlands Trial Register (NTR): NTR2885

Background

Given the growing number of elderly within most western countries, the increasing attention that has been paid to this subpopulation is not surprising. Combined with the relatively high use of care associated with elderly patients, this development makes the elderly an increasingly important subpopulation for healthcare organisations. Studies regarding patient safety have shown that elderly hospitalised patients are at greater risk for preventable adverse events than younger patients [1,2], indicating that this growing group of patients will bring specific requirements and risks to hospitals.

In the Netherlands, the publication of a report regarding adverse events in Dutch hospitals [2,3] led to the start of a national patient safety programme in 2008 [4]. The programme consists of several themes or projects, one of which is the Frail Elderly Project. This project is an evidence-based improvement project aimed at improving care for hospitalised elderly (aged 70 and over). Four health problems are addressed, namely, delirium, falls, malnutrition, and physical impairment. In the project, participating hospitals are advised to screen all elderly patients directly after admission for frailty, defined as the risk of developing or the presence of these four health problems (Additional file 1) [5-8]. Hospitals are provided with advice regarding the prevention and treatment of these problems as well. The overall aim of the Frail Elderly Project is to reduce preventable functional decline in elderly patients caused by hospital stay by improving the quality of care. The national programme provides participating hospitals with advice concerning the project through conferences, the programme website, and a guide. The programme does not provide implementation support in the participating hospitals themselves; implementation of the projects lies solely with the hospitals. In our study, we focus on the Frail Elderly Project, with specific attention to delirium.

Delirium occurs frequently in hospitalised patients, especially in the elderly: about 25% of hospitalised patients aged 65 and over experience a delirium during their hospital stay [9]. Depending on the population that has been studied and the methods that have been used, the delirium rate can increase to 62% (surgical patients) or even 87% (intensive care) [10]. Delirium, sometimes referred to as acute confusional state, is a temporary neuropsychiatric disorder with physiological causes. A delirious patient will experience disturbance of consciousness and a change in cognition or disturbance of perception [10]. The disorder is noted for its acute onset and fluctuating state [9], which are key features that separate delirium from dementia. However, given the fluctuating state of delirium and the similarities in symptoms with dementia, depression, and psychosis, delirium is sometimes hard to diagnose [9,11], which creates a problem for providing adequate delirium care. Studies have indicated that between 32% and 67% of delirium cases were missed during hospital stay [12]. Under-detection and subsequent inadequate care present a problem because delirium in elderly patients is associated with increased length of hospital stay, functional decline, admission to long-term care, and higher mortality [13,14].

Nevertheless, several studies have shown that a significant percentage of delirium cases are preventable, and different strategies have reduced by approximately one-third the incidence of delirious patients [15-18]. The Frail Elderly Project aims to decrease the incidence of delirium in hospitalised elderly patients and minimise the adverse effects associated with delirium, such as functional decline. As mentioned earlier, the implementation of the Frail Elderly Project is a task of the hospitals themselves; no onsite support or training is provided by the national programme. Because several studies have shown that both physicians and nurses have a lack of knowledge concerning delirium and its recognition [9,19,20], an educational tool would appear to be a valuable supplement to the delirium part of the Frail Elderly Project. That an educational intervention can significantly improve delirium recognition and care has been shown by a previous study, which resulted in a strong decline in delirium point prevalence -- 9.8% in the intervention group versus 19.5% in the control group [21].

Because knowledge about delirium is relevant for virtually all nurses working within the hospital, an educational approach that can improve knowledge and skills for large groups of people is needed. E-learning is one method available for efficiently educating a large number of people. A review by Cook et al. [22] has shown that the use of e-learning or 'internet-based education' compared with no intervention is associated with a positive effect on knowledge, skills, and behaviour of healthcare professionals, as well as on patient outcomes. Less clear-cut results have been found for computer-assisted learning aimed at nurses, suggesting more research is needed in this field [23]. Computer-assisted learning refers to all methods of using computerised technology to facilitate education, including e-learning. Whether e-learning can provide a valuable tool for improving delirium care specifically has yet to be determined, though researchers have suggested e-learning as an important option for improving delirium education [24].

The aim of the study is to assess the effects a complementary e-learning course has on the implementation of the delirium part of the Frail Elderly Project, and thus to see whether the use of e-learning improves the provision of delirium care by nurses. The hypothesis underlying this study is that use of e-learning will increase nurses' knowledge regarding delirium and their willingness to change their behaviour regarding at-risk patients. It is hypothesised that these effects will in turn lead to changes in the care provided: screening of elderly patients for risks, observing at-risk patients for delirium, and taking preventative or curative measures. This protocol describes the methodology used in this study and provides details of the stepped wedge cluster randomised controlled trial design [25].

Methods

Intervention

For this study, an e-learning course about delirium will be used that is aimed at nursing staff. This course [26] was developed by a commercial publisher (Noordhoff Publishers), in collaboration with a Dutch hospital. The publisher has been approached by the researchers concerning the use of the e-learning course in this study. Prior to selecting the e-learning for this study, the course was reviewed, while still in development, by the researchers and by the Netherlands Centre of Excellence in Nursing. The content of the e-learning course is consistent with Dutch guidelines regarding delirium care [27,28] and the Frail Elderly Project. The Frail Elderly Project has been implemented by Dutch hospitals independently of this study. The selected e-learning course offers hospitals participating in the study the possibility of supporting their implementation with a course that educates nurses in delirium care (one aspect of the project).

The aims of the e-learning course are on the one hand to create or increase awareness about delirium and the associated risks, and on the other hand to increase knowledge about delirium care. The content of the course consists of information regarding delirium, the risks involved for patients, the recognition of at-risk patients and delirious patients, and the prevention or treatment of delirium. By using fictional patient cases, information is provided in a context familiar to nurses. Throughout the course, users are presented with questions concerning previously given information, providing them with insight into the progress they have made.

During the study, the e-learning course will be offered to nurses in the participating wards online for a period of three months. The estimated time needed per nurse to complete the course, including a knowledge test before and after, is four hours. Nurses can stop the course at any time and continue at a later time. When completing the course and the subsequent knowledge test successfully, nurses will be provided with a certificate stating the completed course and the credits awarded. In cooperation with implementation managers of the publisher, 'kick-off' meetings will be organised in each hospital in order to introduce the e-learning course to the users. The aim of these meetings will be to familiarise users with the way the e-learning works and to encourage them to use and complete the course within the three-month period. After one month and again after two months, users who have not completed the course will receive an e-mail reminder concerning the course to further encourage its use [29]. In addition to these personal reminders, each ward will be provided with a monthly overview of use and completion of the e-learning course.

Setting

Data will be gathered in 18 Dutch hospitals, all of which are at the beginning stages of the implementation of the Frail Elderly Project at the start of the study (self-reported status). These 18 hospitals include two university hospitals, five teaching hospitals, and eleven general hospitals, varying in size (smallest 138 beds, largest 880 beds) and geographical location. One internal medicine ward and one surgical ward of each hospital will participate in the study.

Nearly all hospitals in the Netherlands were initially invited to participate in the study, excluding specialised hospitals such as eye clinics and cancer hospitals, and hospitals already participating in a separate study related to the Frail Elderly Project. The hospitals that responded most quickly to the invitation were included in the study.

Study design

The study will be conducted using a stepped wedge cluster randomised trial design. The defining feature of this design is that the intervention will be introduced to all clusters -- or in this instance, hospitals -- in a sequential order [25,30,31]. At the start of the study in May 2011, no hospital will have access to the intervention, while at the end in March 2012 all hospitals will have been given access. This offers an advantage over parallel trials, in which certain hospitals would not receive the intervention. The e-learning course is expected to do more good than harm, making it unethical to withhold the intervention from participating hospitals [25,31].

After every monthly data collection moment, one or two new hospitals will receive access to the intervention, as illustrated in Figure 1. The start date at which each hospital is given access to the e-learning course will be randomly assigned, using simple or unrestricted randomisation. Hospitals will each be informed of the start date of the intervention prior to the start of the trial. After a period of 11 months, all 18 hospitals will have received the intervention, and the stepped wedge design will have resulted in gathered data from all hospitals for both the control and the intervention group. This offers an important methodological advantage: each hospital acts as its own control, thereby reducing contamination bias [31].

Figure 1.

Figure 1

Open in a new tab

Diagrammatic illustration of the stepped wedge design. Each cell represents a moment of data gathering. The dark blue cells represent data gathering in hospitals without e-learning (control period). The light blue cells represent data gathering in hospitals with e-learning (intervention period).

Data collection

Monthly patient record reviews of patients 70 years or older will be the main source of data. During 11 months, research nurses will review approximately 10 records per ward per month (20 per hospital), using both nursing and medical records. If a ward provides the research nurse with more than 10 records fitting the inclusion criteria, the nurse will take a random sample from the available records. All record reviews will take place within the same week of the month, using records of patients still in hospital or discharged during the previous week. From these records, data concerning provided delirium care will be gathered, which will offer the following information about the implementation of the delirium aspect of the Frail Elderly Project:

1. Detection of risk for delirium, as well as screening for falls, malnutrition, and physical impairment. The Frail Elderly Project has provided hospitals with screening instruments to assess these risks (Additional file 1). Alternative instruments that are used by hospitals are the Groningen Frailty Indicator [32] and the Identification of Seniors At Risk or ISAR [33].

2. Detection of delirium by nurses using the Delirium Observation Screening scale [34,35] (Table 1).

Table 1.

Delirium Observation Screening scale [34,35]

The patient: Never Sometimes or always
1 Dozes during conversation or activities 0 points 1 point
2 Is easily distracted by stimuli from the environment 0 points 1 point
3 Maintains attention to conversation or action 1 point 0 points
4 Does not finish questions or answers 0 points 1 point
5 Gives answers which do not fit the question 0 points 1 point
6 Reacts slowly to instructions 0 points 1 point
7 Thinks to be somewhere else 0 points 1 point
8 Knows which part of the day it is 1 point 0 points
9 Remembers recent events 1 point 0 points
10 Is picking, disorderly, restless 0 points 1 point
11 Pulls IV tubes, feeding tubes, catheters etc. 0 points 1 point
12 Gets easily or suddenly emotional (frightened, angry, irritated) 0 points 1 point
13 Sees persons/things as somebody/something else 0 points 1 point
Open in a new tab

For each of three daily shifts the total score is calculated; the total score per shift is a minimum of 0 and a maximum of 13; the total score for a day is a minimum 0 and a maximum of 39. The DOS scale final score is calculated by dividing the total score for the day by 3; the DOS final score is between 0 and 13

The cut-off point is 3; a DOS scale final score of 3 or more indicates a delirium

3. Diagnosis of delirium by physicians.

4. Prevention of delirium by nursing or medical interventions targeting common reversible causes of delirium.

5. Treatment of delirium by starting nursing or medical interventions targeting common reversible causes of delirium or by prescribing medication.

In addition to data regarding the implementation of the Frail Elderly Project, data regarding the demographic and clinical characteristics of the patients will be gathered.

The record reviews will be carried out by specially trained nurses who have previous experience with reviewing records. An online registration tool will provide these nurses with the means to collect data in the various hospitals. The research nurses will be blinded for the trial condition of the hospitals and are not employed by the hospital where they are conducting the record reviews. Because the e-learning will be used by nurses of the wards participating in the study, the hospitals and wards cannot be blinded to the trial condition.

Besides data from the patient record reviews, data regarding the use of the e-learning course will be gathered. These data include characteristics of the nurses that are given access to the course, such as age, gender, and educational level. Whether the nurses use the course and if so, how often and for how long, will be registered with the web-based course itself. All users will be obligated to complete a delirium knowledge test before and after using the e-learning. These tests consist of randomly selected questions from large question databases. After successfully completing the e-learning and knowledge tests, users will be asked to complete a short evaluation of the course. This evaluation contains five questions regarding the use of the course and the attitude and intention of the user concerning improving delirium care.

Research population

Records of patients admitted to the participating wards will be included in the study if the patient is 70 years or older and if the length of stay at the time of the record review is at least 24 hours. Only records of patients still in hospital at the time of the record review and patients discharged in the previous week will be included. A total of approximately 3,960 patient records will be reviewed in the course of the study.

Sampling

The power calculation for the part of the study focused on the effect of e-learning for nurses on provided care (i.e., screening for delirium risk) resulted in a power of 0.8, based on the following assumptions: 18 hospitals (36 wards); an improvement of delirium risk screening of 20% (from having 40% of the elderly patients screened to 60%) after the e-learning introduction; an alpha of 0.05; a total of 360 patient records for reviewing per month; an intracluster correlation coefficient (ICC) of 1.0.

For the aspect of the study focused on the effect of e-learning on knowledge, we expect a power of 0.99 assuming the study is conducted in 36 wards (18 hospitals), with an e-learning effect size of 1.00 [22], an alpha of 0.05, at least 30 nurses per ward, and an intraclass correlation of 0.20. When adjusting the effect size to 0.5, a power of at least 0.8 is still achieved.

Outcomes and data analysis

In the Frail Elderly Project hospitals are advised to screen all elderly patients directly after admission for the risk of developing or the presence of delirium, falls, malnutrition, or physical impairment. The e-learning course educates nurses on screening for delirium. The percentage of elderly patients that are indeed screened for the risk of delirium is the primary outcome of this study. Secondary outcome measures include: the percentage of elderly patients with an increased risk of delirium that are observed with the Delirium Observation Screening scale; the percentage of elderly patients with (an increased risk of) delirium that receive care aimed to prevent or treat delirium; and the percentage of elderly patients that are diagnosed with delirium. Covariates that will be used in the analysis are: age, gender, living situation, comorbidity [36], and admission diagnosis of the patients.

The data analysis will firstly consist of a comparison between hospitals that have received the intervention and hospitals that have not yet received the intervention, according to the stepped wedge schedule. This will be done using multilevel linear regression and multilevel logistic regression analyses. In these analyses, we will adjust for temporal trends and for clustering within hospitals using generalised estimating equations.

Different organisational or system factors that might influence adherence to the intervention or might lead to differences between hospitals in delivered care will be analysed. These factors include hospital type (academic, teaching, general), size, location (urban or rural), and type of records used (paper versus electronic).

Ethical approval

The Medical Ethics Review Committee of the VU University Medical Centre has approved the study protocol. Patients and representatives on the participating wards will be given the opportunity to object to the use of their (or their relative's) patient record.

Discussion

The number of studies using a stepped wedge design in order to evaluate the effectiveness of an intervention in routine practice is steadily increasing [25]. The reasons for using the design in this study varied: first, it offered an opportunity to introduce the intervention to all participants, which has both ethical and methodological advantages [25,31]. Second, the fact that the intervention will be made available to all participating hospitals will probably have made participating in the study more attractive to hospitals. Third, by using a stepped wedge design the researchers also aim to avoid a problem experienced in other research regarding changes in healthcare: the fact that the context of the study is constantly changing and that any changes made by an intervention can not be separated from changes due to this dynamic context. The stepped wedge design offers an opportunity to take into account the effect of time on the outcome measures and thus to make a distinction between change that would have occurred without the intervention and change attributable to the intervention [25,37].

This study has several potential limitations. The main limitation of the study is that data on provided care and the behaviour of care providers are gathered from patient records. The risk inherent to this method is that changes found are interpreted as actual changes in care and behaviour, while they might only indicate changes in registration. The e-learning course on delirium that nurses will use in this study does not focus on registration however, but educates nurses in improving the care they provide. Because the screening of patients for increased risks can only benefit patients when all relevant care professionals are aware of the outcomes, screening without proper registration would still imply poor quality of care. In this case, an improvement in registration would also suggest an improvement in care. Still, differences in quality of patient records between hospitals or wards might influence the outcome of this study. Records of poor quality could provide research nurses with insufficient information regarding provided care, which could result in the appearance of poor quality of care.

The presence of the research nurse for the monthly record reviews might influence the professionals working on the wards. In order to minimise bias, hospitals will not be provided with information regarding their performance on the Frail Elderly Project during the research period. Another limitation is that the selection of participating hospitals was based on speed of response to the initial invitation. However, the final group of hospitals provide a fairly complete and balanced overview of hospitals within the Netherlands, varying in type of hospital, size, and geographical location. A final limitation is that hospitals were asked to participate in the study if they considered themselves to be at the beginning stages of implementation of the Frail Elderly Project. Which level of implementation had actually been achieved by the hospitals has not been measured or objectively determined before the start of the study. This indicates that the actual situation in each participating hospital regarding the implementation of the project might vary.

Improving quality of care by implementing evidence-based practice is a difficult process. One way to support this process is by providing healthcare professionals with education regarding the desired change. Education can be a vital part of the implementation process, especially for complex health issues such as delirium. E-learning provides healthcare organisations with a potentially effective and simple way of educating large groups of professionals. This study will contribute to a growing body of knowledge concerning e-learning and the effects it can have on knowledge as well as delivered care.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

CW, ML, and LS collectively developed the study plan. CW conceived of the study, and led the application for current funding through the Dutch Ministry of Health, Welfare and Sport. LS has drafted the initial form and final revision of this manuscript. All authors (CW, ML and RI) have read drafted components of the manuscript, provided input into initial and final refinements of the total manuscript, and agreed to the final manuscript.

Supplementary Material

Additional file 1

Screening instruments Frail Elderly Project.

Click here for file (34KB, DOC)

Contributor Information

Lotte Van de Steeg, Email: l.vandesteeg@nivel.nl.

Maaike Langelaan, Email: m.langelaan@nivel.nl.

Roelie Ijkema, Email: r.ijkema@nivel.nl.

Cordula Wagner, Email: c.wagner@nivel.nl.

References

  1. Thomas EJ, Brennan TA. Incidence and type of preventable adverse events in elderly patients: population based review of medical records. BMJ. 2000;320(7237):741–744. doi: 10.1136/bmj.320.7237.741. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Zegers M, de Bruijne MC, Wagner C, Hoonhout LHF, Waaijman R, Smits M, Hout FAG, Zwaan L, Christiaans-Dingelhoff I, Timmermans DRM, Groenewegen PP, Van der-Wal G. Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care. 2009;18(4):297–302. doi: 10.1136/qshc.2007.025924. [DOI] [PubMed] [Google Scholar]
  3. de Bruijne MC, Zegers M, Hoonhout LHF, Wagner C. Onbedoelde schade in Nederlandse ziekenhuizen: dossieronderzoek van ziekenhuisopnames in 200. Amsterdam/Utrecht: EMGO+/NIVEL; 2007. [Google Scholar]
  4. Over het VMS Veiligheidsprogramma. Dutch national patient safety programme http://www.vmszorg.nl/Over-het-programma.
  5. Kwetsbare ouderen. http://www.vmszorg.nl/Documents/Tools_Extras/Thema's/KO/web_2009.0104_praktijkgids_kwetsbare_ouderen.pdf Dutch patient safety programme.
  6. Kruizenga HM, Seidell JC, de Vet HCW, Wierdsma NJ, van Bokhurst-de van der Schueren MAE. Development and validation of a hospital screening tool for malnutriton: the short nutritional assessment questionnaire (SNAQ) Clin Nutr. 2005;24(1):75–82. doi: 10.1016/j.clnu.2004.07.015. [DOI] [PubMed] [Google Scholar]
  7. Stratton RJ, Hackston A, Longmore D, Dixon R, Price S, Stroud M, King C, Elia M. Malnutrition in hospitalised outpatients and inpatients: prevalence, concurrent validity and ease of use of the 'malnutrition universal screening tool' ('MUST') for adults. Br J Nutr. 2004;92(5):799–808. doi: 10.1079/BJN20041258. [DOI] [PubMed] [Google Scholar]
  8. Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. The index of ADL: a standardized measure of biological and psychosocial function. JAMA. 1963;185:914–919. doi: 10.1001/jama.1963.03060120024016. [DOI] [PubMed] [Google Scholar]
  9. Schuurmans MJ, Duursma SA, Shortridge-Baggett LM. Early recognition of delirium: review of the literature. J Clin Nurs. 2001;10(6):721–729. doi: 10.1046/j.1365-2702.2001.00548.x. [DOI] [PubMed] [Google Scholar]
  10. Mittal V, Muralee S, Williamson D, McEnerney N, Thomas J, Cash M, Tampi RR. Delirium in the elderly: a comprehensive review. Am J Alzheimers Dis Other Demen. 2001;26(2):97–109. doi: 10.1177/1533317510397331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Inouye SK. Delirium in older persons. N Engl J Med. 2006;354(11):1157–1165. doi: 10.1056/NEJMra052321. [DOI] [PubMed] [Google Scholar]
  12. Inouye SK. The dilemma of delirium: clinical and research controversies regarding diagnosis and evaluation of delirium in hospitalized elderly medical patients. Am J Med. 1994;97(3):278–288. doi: 10.1016/0002-9343(94)90011-6. [DOI] [PubMed] [Google Scholar]
  13. Anderson D. Preventing delirium in older people. Br Med Bul. 2005;73-74:25–34. doi: 10.1093/bmb/ldh048. [DOI] [PubMed] [Google Scholar]
  14. Witlox J, Eurelings LSM, de Jonghe JFM, Kalisvaart KJ, Eikelenboom P, van Gool WA. Delirium in elderly patients and the risk of postdischarge mortality, institutionalization and dementia. JAMA. 2010;304(4):443–451. doi: 10.1001/jama.2010.1013. [DOI] [PubMed] [Google Scholar]
  15. Cole MG. Delirium: effectiveness of systematic interventions. Dement Geriatr Cogn Disord. 1999;10(5):405–411. doi: 10.1159/000017179. [DOI] [PubMed] [Google Scholar]
  16. Inouye SK, Bogardus ST, Charpentier PA, Leo-Summers L, Acampora D, Holford TR, Cooney LM. A multicomponent intervention to prevent delirium in hospitalized older patients. N Engl J Med. 1999;340(9):669–676. doi: 10.1056/NEJM199903043400901. [DOI] [PubMed] [Google Scholar]
  17. Marcantonio ER, Bergmann MA, Kiely DK, Orav EJ, Jones RN. Randomized trial of a delirium abatement program for postacute skilled nursing facilities. J Am Geriatr Soc. 2010;58(6):1019–1026. doi: 10.1111/j.1532-5415.2010.02871.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Vidán MT, Sánchez E, Alonso M, Montero B, Ortiz J, Serra JA. An intervention integrated into daily clinical practice reduces the incidence of delirium during hospitalization in elderly patients. J Am Geriatr Soc. 2009;57(11):2029–2036. doi: 10.1111/j.1532-5415.2009.02485.x. [DOI] [PubMed] [Google Scholar]
  19. Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM. Nurses' recognition of delirium and its symptoms. Arch Intern Med. 2001;161(20):2467–2473. doi: 10.1001/archinte.161.20.2467. [DOI] [PubMed] [Google Scholar]
  20. Agar M, Draper B, Philips PA, Philips J, Collier A, Harlum J, Currow D. Making decisions about delirium: a qualitative comparison of decision making between nurses working in palliative care, aged care, aged care psychiatry, and oncology. Palliat Med. 2011. in press . [DOI] [PubMed]
  21. Tabet N, Hudson S, Sweeney V. An educational intervention can prevent delirium on acute medical wards. Age Ageing. 2005;34(2):152–156. doi: 10.1093/ageing/afi031. [DOI] [PubMed] [Google Scholar]
  22. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet- based learning in the health professions: a meta-analysis. JAMA. 2008;300(10):1181–1196. doi: 10.1001/jama.300.10.1181. [DOI] [PubMed] [Google Scholar]
  23. Bloomfield JG, While AE, Robert JD. Using computer assisted learning for clinical skills education in nursing: integrative review. J Adv Nurs. 2008;63(3):222–235. doi: 10.1111/j.1365-2648.2008.04653.x. [DOI] [PubMed] [Google Scholar]
  24. Irving K, Detroyer E, Formean M, Milisen K. The virtual gateweay: opening doors in delirium teaching and learning. Int Rev Psychiatry. 2009;21(1):15–19. doi: 10.1080/09540260802674752. [DOI] [PubMed] [Google Scholar]
  25. Mdege ND, Man M, Taylor CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly use to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64(9):936–948. doi: 10.1016/j.jclinepi.2010.12.003. [DOI] [PubMed] [Google Scholar]
  26. Cursus "Delier": Preventie en zorg in het ziekenhuis. Noordhoff Publishers; http://www.campus-med.nl/delier.html [Google Scholar]
  27. Richtlijn delirium. Dutch Institute for Healthcare Improvement http://www.cbo.nl/Downloads/208/delirium_rl_2005.pdf.
  28. Basisset kwaliteitsindicatoren ziekenhuizen 2011. Dutch Health Care Inspectorate. http://www.igz.nl/Images/2010-07%20Basisset%20kwaliteitsindicatoren%20ziekenhuizen%202011_tcm294-283436.pdf.
  29. Abdolrasulnia M, Collns BC, Casebeer L, Wall T, Spettell C, Ray MN, Weissman NW, Allison JJ. Using e-mail reminders to engage physicians in an Internet-based CME intervention. BMC Med Educ. 2004;4:17. doi: 10.1186/1472-6920-4-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–191. doi: 10.1016/j.cct.2006.05.007. [DOI] [PubMed] [Google Scholar]
  31. Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54. doi: 10.1186/1471-2288-6-54. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Steverink N, Slaets JPJ, Schuurmans H, Van-Lis M. Measuring Frailty. Development and testing of the Groningen Frailty Indicator (GFI) Gerontologist. 2001;41:236–237. [Google Scholar]
  33. McCusker J, Bellavance F, Cardin S, Trépanier S, Verdon J, Ardman O. Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc. 1999;47(10):1229–1237. doi: 10.1111/j.1532-5415.1999.tb05204.x. [DOI] [PubMed] [Google Scholar]
  34. Schuurmans MJ. PhD thesis. University Utrecht, Division of Nursing Science; 2001. Early recognition of delirium. [Google Scholar]
  35. van Gemert LA, Schuurmans MJ. The Neecham Confusion Scale and the Delirium Observation Screening Scale: capacity to discriminate and ease of use in clinical practice. BMC Nurs. 2007;6:3. doi: 10.1186/1472-6955-6-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
  37. Moulton LH, Golub JE, Durovni B, Cavalcante SC, Pacheco AG, Saraceni V, King B, Chaisson RE. Statistical design of THRio: a phased implementation clinic- randomized study of a tuberculosis preventive therapy intervention. Clin Trails. 2007;4(2):190–199. doi: 10.1177/1740774507076937. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Additional file 1

Screening instruments Frail Elderly Project.

Click here for file (34KB, DOC)

Articles from Implementation Science : IS are provided here courtesy of BMC

RESOURCES