Quality of clinical practice guidelines in delirium: a systematic appraisal

Abstract

Objective

To determine the accessibility and currency of delirium guidelines, guideline summary papers and evaluation studies, and critically appraise guideline quality.

Design

1. Systematic literature search for formal guidelines (in English or French) with focus on delirium assessment and/or management in adults (≥18 years), guideline summary papers and evaluation studies.

2. Full appraisal of delirium guidelines published between 2008 and 2013 and obtaining a ‘Rigour of Development’ domain screening score cut-off of >40% using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument.

Data sources

Multiple bibliographic databases, guideline organisation databases, complemented by a grey literature search.

Results

3327 database citations and 83 grey literature links were identified. A total of 118 retrieved delirium guidelines and related documents underwent full-text screening. A final 21 delirium guidelines (with 10 being >5 years old), 12 guideline summary papers and 3 evaluation studies were included. For 11 delirium guidelines published between 2008 and 2013, the screening AGREE II ‘Rigour’ scores ranged from 3% to 91%, with seven meeting the cut-off score of >40%. Overall, the highest rating AGREE II domains were ‘Scope and Purpose’ (mean 80.1%, range 64–100%) and ‘Clarity and Presentation’ (mean 76.7%, range 38–97%). The lowest rating domains were ‘Applicability’ (mean 48.7%, range 8–81%) and ‘Editorial Independence’ (mean 53%, range 2–90%). The three highest rating guidelines in the ‘Applicability’ domain incorporated monitoring criteria or audit and costing templates, and/or implementation strategies.

Conclusions

Delirium guidelines are best sourced by a systematic grey literature search. Delirium guideline quality varied across all six AGREE II domains, demonstrating the importance of using a formal appraisal tool prior to guideline adaptation and implementation into clinical settings. Adding more knowledge translation resources to guidelines may improve their practical application and effective monitoring. More delirium guideline evaluation studies are needed to determine their effect on clinical practice.

Strengths and limitations of this study.

• A strength of this study is the usage of an experienced university health sciences librarian to assist with the search strategy for the systematic literature search.

• A complement of four (rather than two) appraisers conducted the full Appraisal of Guidelines for Research and Evaluation (AGREE II) quality assessment of the delirium guidelines.

• It was not possible to retrieve the full text for all literature search citations.

• Language was restricted to English and French for the second-level screening.

• Two delirium guideline updates have been published since the initial literature search.

Introduction

Delirium is a common neurocognitive disorder with the hallmark of disturbed attention and awareness developing over a short period of time.¹ Common predisposing risk factors for delirium are advanced age and dementia. Delirium prevalence in general medical hospital settings is 18–35%,^{2 3} 26–62% in palliative care units,⁴ and 30–70% in critical care, depending on the patient population and assessment methods.^{5 6} Prevalence rates of up to 88% at the end of life (last hours to weeks of life) have been reported.^{4 7} However, the diagnosis of delirium is frequently missed or misdiagnosed in multiple clinical settings due to fluctuating symptoms and signs, in addition to a lack of routine cognitive screening and assessment of attention.^{2 8–12} Delirium causes functional impairment, increased falls, increased healthcare costs, prolonged hospitalisation with an increased risk of placement in long-term care at discharge and increased risk of mortality.^{13 14} It worsens pre-existing dementia and increases the risk for developing de novo dementia.¹⁵ Delirium also causes significant psychological distress for patients, families and healthcare providers.¹⁶ It has been estimated that ∼30% of delirium episodes can be prevented and treated using multicomponent non-pharmacological strategies.^{2 17} Delirium is usually multifactorial in origin, and is potentially reversible if the identified precipitant/s are treatable. Although the current mainstay of delirium treatment is supportive non-pharmacological care, pharmacological treatment of delirium with antipsychotics has historically been used despite limited evidence for efficacy and concerns regarding their adverse effects, especially in patients with pre-existing dementia.^18–23 It should also be noted that currently no antipsychotic has an official indication for delirium. Various pharmacological agents have also been trialled as delirium preventative strategies.^{2 24} The multidimensional management of delirium requires interprofessional comprehensive care. With significant ongoing patient morbidity and healthcare costs due to delirium, there remains a pressing need to improve the overall management of this deleterious clinical syndrome for patients and their families.

Shaneyfelt stated that clinical practice guidelines should enhance healthcare quality and outcomes for many conditions.²⁵ Guidelines may potentially assist clinicians with clinical care and decision-making, and provide standardisation of care delivery across disciplines within an institution as well as interinstitutional benchmarking. Clinical practice guidelines have been defined as ‘systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances’.²⁶ More recently, the Canadian Medical Association handbook on clinical practice guidelines stated that guidelines aim to ‘summarise research findings and make clinical decisions more transparent’ and ‘identify gaps in knowledge and prioritise research activities’.²⁷ Recently, authors have questioned the trustworthiness of guidelines.^{25 28} Only applicable, high-quality, rigorous, non-biased guidelines with valid recommendations should be selected by clinicians to adapt and implement into their clinical practice.^29–31

Multiple clinical practice guidelines on delirium have been published despite a lack of high-level evidence for the management of delirium, with the result that many final guideline recommendations are derived from ‘expert’ opinion.³² Our previous non-systematic search for formal delirium guidelines demonstrated that guidelines were difficult to source using selected databases.³² We subsequently sought to determine and review rigorous national and international delirium guidelines that would be applicable for adaptation and implementation into a palliative care setting, as well as develop a site-specific evaluation strategy. To the best of our knowledge, the quality and validity of published delirium guidelines have not been reported. To appraise their currency and accessibility, we conducted a systematic search for delirium guidelines published between 1990 and 2013. Using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument,³³ a critical assessment of the quality and validity of guidelines published from 2008 to 2013 was performed. In addition, guideline summaries and studies evaluating implemented guidelines were identified.

Methods

Data sources and searches

A systematic literature search for existing delirium clinical practice guidelines for any clinical setting, summary papers and studies evaluating implemented guidelines was conducted in multiple electronic bibliographic databases by an information specialist librarian. The search for the Cochrane Library and the Database of Reviews of Effects (DARE), Medline (Ovid), EMBASE (Ovid), CINAHL, PsycINFO (Ovid) and TRIP (Turning Research into Practice) databases was executed on 16 August 2013 with no date limitations, and no language or age restriction. Search terms included ‘delirium’, ‘guideline’, ‘health planning guideline’, and ‘Clinical protocols’. Box 1 shows the search strategy for Medline (Ovid).

Box 1. Methodology for systematic appraisal of delirium clinical practice guidelines.

Medline (Ovid) search strategy:

1. Delirium/

2. deliri*.tw.

3. (acute adj1 confusion).tw.

4. (acute adj1 brain adj1 syndrome).tw.

5. (metabolic adj1 encephalopath*).tw.

6. (acute adj1 organic adj1 psychosyndrome).tw.

7. (acute adj1 psycho-organic adj1 syndrome).tw.

8. (clouded adj1 state).tw.

9. (clouding adj3 consciousness).tw.

10. (exogenous adj1 psychos*).tw.

11. (toxic adj1 confusion).tw.

12. (toxic adj1 psychos*).tw.

13. Confusion/

14. (acute adj1 confusional adj1 state).tw.

15. (acute adj1 brain adj1 failure).tw.

16. (terminal* adj1 restless*).tw.

17. (terminal adj1 agitation).tw.

18. (psychomotor adj1 agitation).tw.

19. (cognitive adj1 failure).tw.

20. disorientation.tw.

21. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18 or 19 or 20

22. exp guideline/

23. exp Guidelines as Topic/

24. guideline*.pt.

25. practice guideline.pt.

26. Health Planning Guidelines/

27. Clinical Protocols/

28. Critical Pathways/

29. (practice adj3 paramet*).tw.

30. (clinical adj3 pathway*).tw.

31. consensus development conference.pt.

32. consensus development conference nih.pt.

33. 22 or 23 or 24 or 25 or 26 or 27 or 28 or 29 or 30 or 31 or 32

34. 21 and 33

Guideline databases:

National Guideline Clearinghouse (http://www.guideline.gov), Guidelines International Network (http://www.g-i-n.net), National Institute for Clinical Excellence (http://www.nice.org.uk)

Scottish Intercollegiate Guidelines Network (http://www.sign.ac.uk), New Zealand Guidelines Group (http://www.nzgg.org.nz)

Canadian Medical Association—CMA Infobase: Clinical Practice Guidelines (http://www.cma.ca/cpgs/)

Other databases:

CareSearch (http://www.caresearch.com.au/Caresearch/Default.aspx), Centre for Effective Practice (http://www.effectivepractice.org)

Delirium associations:

European Delirium Association (http://www.europeandeliriumassociation.com), American Delirium Society (http://www.americandeliriumsociety.org)

For this project, we used the CAN-IMPLEMENT Resource, a detailed guideline adaptation and implementation planning resource that was developed by the Canadian Guideline Adaptation Study Group. As per CAN-IMPLEMENT recommendations,³⁴ a search of guideline organisation databases, including the USA National Guideline Clearinghouse (NGC)³⁵ and the Guidelines International Network (G-I-N) International Guideline Library,³⁶ was also performed. Using the aforementioned search terms, a grey literature search was performed (between 9 May and 20 December 2013) through Google and Google Scholar, as well as reviewing the websites of relevant organisations, such as international delirium associations. Any possible documents related to delirium guidelines (such as summary versions and pathways) found on the grey literature search were retrieved. Supplemental hand searching for delirium guidelines was also conducted.

Appraisal of Guidelines for Research and Evaluation II instrument (AGREE II)

The AGREE II instrument appraises the quality of guidelines by focusing on the guideline development process, as opposed to formally evaluating the content of a guideline. It consists of 23 items in six distinct domains: ‘Scope and Purpose’, ‘Stakeholder Involvement’, ‘Rigour of Development’, ‘Clarity and Presentation’, ‘Applicability’ and ‘Editorial Independence’. Each AGREE II item is scored on a 7-point scale from 1= strongly disagree to 7= strongly agree. An additional two global rating items at the end of AGREE II provide an overall assessment of the guideline. Domain 3 of AGREE II, the ‘Rigour of Development’ domain, has eight items and assesses the ‘process of gathering and summarising the evidence, and methods used to develop recommendations’.³³ Within the ‘Rigour of Development’ domain, AGREE II item number 10 considers that ‘the health benefits, side effects and risks have been considered in formulating the recommendations’. The 4-item Domain 5, known as ‘Applicability’, ‘pertains to the likely barriers and facilitators to implementation, strategies to improve uptake and resource implications of applying the guideline’.³³ The online version of the AGREE II instrument was used to appraise the quality of guidelines after all appraisers had completed standardised online training.³³

Delirium clinical practice guideline selection

The retrieved citations and internet sources were initially independently screened by two appraisers (SB, KM) according to title and abstract. Inclusion criteria included: full and short versions of formal clinical practice guidelines, as defined by Field and Lohr (see above),²⁶ with a focus on the topic of delirium and in any clinical setting for adults ≥18 years, guideline summary papers and studies evaluating guidelines. Citations were excluded if the scope of the paper was on alcohol withdrawal delirium (delirium tremens), paediatric population only or literature reviews on delirium management (systematic or otherwise). Duplicates were included at this stage to identify individual citation numbers per database.

Citations meeting the above inclusion criteria went through a second-level screening of full-text articles and internet sources (SB, KM) to determine the delirium clinical practice guidelines to be assessed, with incongruities regarding a citation's eligibility being resolved by consensus. Guidelines to be assessed were defined as formal delirium guidelines, or clinical practice guidelines, with a focus on delirium assessment and/or management in an adult population (≥18 years) in either English or French language, ‘produced by a sanctioned legitimate group’ (ie, a formal endorsed guideline development committee or task force, as opposed to an individual contributor) with a systematic literature search and including clear evidence-informed recommendations linked to defined levels of evidence,³⁴ and published within one of the following three time periods: 2008–2013, 2003–2007, older than 2003. Guidelines that solely focused on the management of postoperative delirium were excluded as the authors' aim was to review guidelines that were most applicable to patients in a medical inpatient or palliative care setting. Delirium guidelines published between the years 2008 and 2013 were determined to be ‘high priority’ guidelines due to currency, thereby matching the NGC inclusion criteria that require its included guidelines to have been ‘developed, reviewed or revised within the previous five years’.³⁵

Two appraisers (SB, KM) independently assessed delirium guidelines published between 2008 and 2013 using the ‘Rigour’ domain of the AGREE II instrument, as recommended by CAN-IMPLEMENT V.3.1.^{34 37} Differences were resolved by consensus. To include a local provincial delirium guideline, guidelines meeting an AGREE II ‘Rigour of Development’ domain cut-off of >40% were submitted to the next step in quality assessment.

Quality assessment of delirium guidelines

Four members of the research team (SB/KM/MA/DD) independently appraised the final included guidelines using the full 23-item AGREE II instrument, followed by a teleconference consensus discussion among all four appraisers. Standardised domain scores for each of the six AGREE II domains were calculated as recommended by AGREE II. In addition, the appraisers added contemporaneous field notes and completed the two global rating items at the end of each AGREE II assessment. The first global rating item asks appraisers to rate the overall quality of the guideline on a 7-point scale (1= lowest possible quality and 7= highest possible quality). The second global rating item asks whether the appraiser would recommend the guideline for use in practice, with options of ‘yes’, ‘yes, with modifications’, and ‘no’.

Results

Systematic search for delirium clinical practice guidelines

The systematic search generated 3327 citations, of which 1629 were for the search strategy time period 2008–2013. Ninety-three database citations related to guidelines met the inclusion criteria (see figure 1). Second-level screening of full-text articles assessed some articles with the word ‘guideline’ in the title as literature reviews or protocols for delirium management without clear links between level of evidence and recommendations; these were excluded. The grey literature search of websites resulted in 21 additional documents related to ‘guidelines’. The search of the global G-I-N Library using the term ‘delirium’ found 77 results out of 6444 entries. Seven entries appeared to meet the inclusion criteria of which only one had been retrieved from the above search.³⁸ It was not possible to retrieve the other six possible guidelines (all non-English: four written in Dutch, one in French and one in German) as no research team member had G-I-N membership.^39–44 With the addition of four delirium guidelines found on hand searching, a total of 118 retrieved guidelines and documents related to guidelines underwent full-text screening for eligibility.

Figure 1.

Flow diagram of delirium guideline selection procedure.

Twenty-one stand-alone delirium guidelines were identified from the second-level screening; 11 published between 2008 and 2013, 7 between 2003 and 2007, and 3 guidelines published before 2003 (see tables 1 and 2).

Table 1.

AGREE II initial ‘rigour’ domain scores for delirium clinical practice guidelines (published 2008–2013)

Clinical practice guideline name	Guideline development group	Year of publication	Country/language	Database source	Target patient population	Initial Rigour domain score
Delirium: diagnosis, prevention	National Clinical Guideline Centre (NCGC): Commissioned by National Institute for Health and Care Excellence (NICE)	2010	England: English	Grey literature, TRIP; Guideline summary (NGC): TRIP	Adult patients ≥18 years in a hospital setting and long-term residential care	91%
and management. Clinical Guideline 103 (CG103)38
Clinical practice guidelines for the management of pain, agitation and delirium in adult patients in the intensive care unit45	The American College of	2013	USA: English	Medline, EMBASE, CINAHL; Guideline summary (NGC): TRIP	Intubated and non-intubated adult medical, surgical and trauma intensive care unit patients	58%
	Critical Care Medicine task force
Caregiving strategies for older adults with delirium, dementia and depression46	Registered Nurses Association of Ontario (RNAO)	2010 update (Original: 2004)	Canada: English	Grey literature, TRIP; Guideline summary (NGC): TRIP	Older adults ≥65 years with delirium, dementia and/or depression	55%
Guideline on the assessment and treatment of delirium in older adults at the end of life47	Canadian Coalition for Seniors' Mental Health (CCSMH)	2010	Canada: English	Grey literature	Older adults ≥65 years with or at risk of developing delirium and approaching the end of life	54%
Screening for delirium, dementia and depression in older adults48	Registered Nurses Association of Ontario (RNAO)	2010 update (Original: 2003)	Canada: English	Grey literature, TRIP; Guideline summary (NGC): TRIP	Older adult clients	48%
Cancer Care Ontario's Symptom management guide-to-practice: Delirium49	Cancer Care Ontario (CCO)	2010	Canada: English	Grey literature	Adult patients who require symptom management related to cancer	43%
Acute Confusion/Delirium50	Sendelbach S, Finch Guthrie P. (University of Iowa College of Nursing)	2009	USA: English	Hand search; Guideline summary (NGC): TRIP	Elderly hospitalised patients (general medical/surgical units)	42%
Evidence and consensus-based German guidelines for the management of analgesia, sedation and delirium in intensive care51	Association of Scientific Medical Societies (AWMF) of Germany	2010	Germany: English (Original version)/German	Medline	Critically ill patients	21%
Palliative Care. NCCN Guidelines52	National Comprehensive Cancer Network (NCCN)	2013	USA: English	Grey literature	Adult cancer patients	9%
Clinical guideline: The prevention, recognition and management of delirium in adult inpatients53	Guy's and St. Thomas’ NHS Foundation Trust	2013	England: English	Hand search	Hospitalised adult patients	9%
British Columbia guidelines: Palliative care for the patient with incurable cancer or advanced disease. Part 2: Pain and symptom management. Delirium management54	Family Practice Oncology Network and the Guidelines and Protocols Advisory Committee	2011	Canada: English	Grey literature; Guideline summary (NGC): TRIP	Patients with cancer or advanced disease	3%

NGC, National Guideline Clearinghouse; TRIP, Turning Research into Practice.

Table 2.

Summary of delirium clinical practice guidelines published (1) between 2003 and 2007 (2) before 2003

Delirium guideline name	Guideline development Group	Year	Country	Language	Database source	Target patient population
Delirium: Guidelines for general hospitals55	The Delirium Guidelines Development Group ‘A multidisciplinary expert panel’	2007	Switzerland	English	Medline, EMBASE, PsycINFO	Adult patients in general hospitals
Clinical practice guidelines for the management of delirium in older people56	Clinical Epidemiology and Health Service Evaluation Unit, Melbourne Health in collaboration with the Delirium Clinical Guidelines Expert Working Group	2006	Australia	English	Grey literature	Older people receiving care (in hospital, community and residential care settings) aged ≥65 years, or ≥45 years in the case of Aboriginal and Torres Strait Islander people
National guidelines for seniors’ mental health: The assessment and treatment of delirium57	Canadian Coalition for Seniors’ Mental Health (CCSMH)	2006	Canada	English/French	Grey literature	Older adults ≥65 years, all healthcare settings
Symptom Guidelines: Delirium/Restlessness58	Hospice Palliative Care Clinical Practice Guideline Committee, Fraser Health	2006	Canada	English	Grey literature	Adult patients ≥19 years living with advanced life-threatening illness
Guidelines for the prevention, diagnosis and management of delirium in older people. Concise guidance to good practice series, No 659	Guideline Development Group: British Geriatrics Society and Royal College of Physicians	2006	UK	English	Hand search	Older people in hospital and community care settings
Detection, prevention and treatment of delirium in critically ill patients60	UK Clinical Pharmacy Association	2006	UK	English	Hand search	Critically ill patients
(Guideline: Delirium)61	Dutch Psychiatric Association. van der Mast RC et al	2005	Netherlands	Dutch	Medline, EMBASE	Not known due to language restriction
Clinical practice guidelines for the sustained use of sedatives and analgesics in the critically ill adult62	Sedation and Analgesia Taskforce, Society of Critical Care Medicine and American Society of Health-System Pharmacists	2002	USA	English	EMBASE	Critically ill adults
Clinical practice guideline: Delirium in the elderly63	Fraser Health Authority	2002	Canada	English	Grey literature	Elderly patients with delirium
Practice guideline for the treatment of patients with delirium64	American Psychiatric Association	1999	USA	English	Medline, EMBASE, PsycINFO, Grey literature	Patients with delirium

The initial ‘Rigour of Development’ domain scores for the 11 guidelines ≤5 years old from the two appraisers ranged from 3% to 91% (see table 1). To include a local provincial delirium guideline,⁴⁹ a ‘Rigour’ domain cut-off score of >40% was used, resulting in a final seven ‘high priority’ guidelines requiring further evaluation, and four excluded guidelines.^51–54

The final seven ‘high priority’ guidelines from 2008 to 2013 were all written in English, with four having been developed in Canada, two in the USA and one in the UK. Both guidelines from the Registered Nurses Association of Ontario (RNAO) were originally published in 2003: we evaluated the revised 2010 supplements. The 2009 full version of the guideline by Sendelbach et al (updated from 1998) was purchased from the University of Iowa after sourcing it via a website link on the Medline-retrieved summary guideline paper. The target users were healthcare providers for three guidelines,^{38 47 50} healthcare providers providing care to patients with cancer-related symptom management throughout the disease trajectory for one guideline,⁴⁹ nurses (Registered Nurses and Registered Practical Nurses) for two guidelines^{46 48} and intensive care unit (ICU) clinicians for one guideline.⁴⁵ Four guidelines were limited to older adults,^{46–48 50} although only two provided an age definition for ‘older adult’. The 2010 Clinical Guideline 103 (CG103) on delirium (with an ‘Evidence Update’ in April 2012 which was also reviewed by the appraisers) from the National Clinical Guideline Centre (NCGC) was commissioned by the National Institute for Health and Clinical Excellence (NICE).³⁸ It was the longest guideline, comprising 447 pages in its full version in addition to 10 appendices, implementation tools and care pathway.

The NICE guideline specifically excluded ‘people receiving end-of-life care’. The guideline by the Canadian Coalition for Seniors' Mental Health (CCSMH) was developed specifically for palliative care patients and defined ‘approaching the end of life’ as ‘to living currently with a terminal illness and having an estimated life expectancy of six months or less’.⁴⁷ The ICU guideline (for the management of pain, agitation and delirium or ‘PAD’) was published by Barr et al⁴⁵ as a special article in a specialist journal, rather than as a stand-alone guideline. This meant that while it was easily sourced on searching electronic databases, such as Medline, CINAHL and EMBASE, the initial version of the guideline had to be cut by 10–20% due to journal space restrictions.⁶⁵ Three guidelines searched for pre-existing guidelines from which to make their recommendations, as opposed to being de novo guidelines.^{46 48 49}

Quality appraisal of selected guidelines (published between 2008 and 2013) using the AGREE II instrument

Table 3 summarises the AGREE II domain scores and overall results of the quality assessment.

Table 3.

Final AGREE II domain scores and global ratings

Delirium guideline name	Domain 1: Scope and Purpose	Domain 2: Stakeholder Involvement	Domain 3: Rigour of Development	Domain 4: Clarity of Presentation	Domain 5: Applicability	Domain 6: Editorial Independence	Overall global rating scale (average)	Overall recommendation (to use guideline)
Delirium: diagnosis, prevention and management (NICE Clinical Guideline 103 (CG103))38	100%	85%	93%	97%	81%	85%	6.75	Yes (3 reviewers) Yes with modifications (1 reviewer)
Clinical practice guidelines for the management of pain, agitation and delirium in adult patients in the intensive care unit45	96%	56%	78%	93%	60%	90%	6.25	Yes (4 reviewers)
Caregiving strategies for older adults with delirium, dementia and depression46	86%	58%	65%	85%	69%	46%	6.75	Yes (2 reviewers) Yes with modifications (2 reviewers)
Guideline on the assessment and treatment of delirium in older adults at the end of life47	76%	65%	58%	74%	8%	44%	4.5	Yes (1 reviewer) Yes with modifications (3 reviewers)
Screening for delirium, dementia and depression in older adults48	71%	60%	61%	79%	72%	48%	5	Yes (3 reviewers) Yes with modifications (1 reviewer)
Cancer Care Ontario's Symptom management guide-to-practice: Delirium49	68%	58%	47%	71%	16%	56%	3.75	Yes (1 reviewer) Yes with modifications (3 reviewers)
Acute Confusion/Delirium50	64%	28%	34%	38%	35%	2%	3.5	No (2 reviewers) Yes with modifications (2 reviewers)
Mean	80.1%	58.6%	62.3%	76.7%	48.7%	53.0%
Range	64–100%	28–85%	34–93%	38–97%	8–81%	2–90%

The seven selected ‘high priority’ delirium guidelines scored highest for the ‘Scope and Purpose’ domain (mean 80.1%, range 64–100%); and the ‘Clarity and Presentation’ domain (mean 76.7%, range 38–97%). Overall the lowest rating domains were Domain 5, ‘Applicability’ (mean 48.7%, range 8–81%), and Domain 6, ‘Editorial Independence’, (mean 53%, range 2–90%). The mean values for the remaining two domains were 62.3% (‘Rigour of Development’) and 58.6% (‘Stakeholder Involvement’). The 2010 NCGC/NICE guideline was the highest rated guideline in five of the six domains (range 81–100%). Only the ‘PAD’ ICU guideline, which was rated the highest for ‘Editorial Independence’, was recommended for use without modifications by all four appraisers.

The RNAO guidelines received the highest ratings for monitoring criteria (presented in the original 2003 and 2004 guidelines) and provided a link for a separate comprehensive Toolkit (available in English and French) for implementing guidelines.⁶⁶ The RNAO screening guideline had been pilot tested at three teaching hospitals, resulting in a summary of implementation strategies within the guideline. The NICE guideline had a costing report and costing template available, in addition to a brief implementation advice document and audit template. From the contemporaneous field notes, it was noted that the ‘PAD’ ICU guideline did not rely on expert opinion and explicitly made no recommendation if insufficient evidence or no group consensus reached. Two appraisers thought that the length of the full NICE guideline may impede implementation, but noted that the guideline had excellent summary documents including a 29-page shorter version and a 10-page ‘Quick reference guide’. One appraiser found it challenging to navigate through PDF versions of several guidelines without running titles to define sections.

Systematic search for guideline summaries, dissemination papers and studies

For the years 2008–2013, four of the final seven appraised guidelines had published dissemination papers, with the NICE guideline having two summary publications^{17 67} (see table 4). All the dissemination papers were sourced through Medline.

Table 4.

Retrieved dissemination summary papers for delirium guidelines (2008–2013)

Summary paper title	Author(s)	Year of publication	Database source (eg, Medline, Google)	Guidelines referred to; year of guideline publication
Clinical practice guidelines for the management of pain, agitation and delirium in adult patients in the intensive care unit: Executive summary68	Barr et al	2013	Medline, CINAHL, EMBASE	Clinical practice guidelines for the management of pain, agitation and delirium in adult patients in the intensive care unit; 201345
Developing guidelines on the assessment and treatment of delirium in older adults at the end of life69	Brajtman et al	2011	Medline	Guideline on the assessment and treatment of delirium in older adults at the end of life; 201047
Das Delir—Konsequenzen für die Analgosedierung kritisch kranker Patienten (ICU delirium: Consequences for management of analgesia and sedation in the critically ill)70	Lütz and Spies (In German)	2011	Medline, EMBASE	Analgesie, Sedierung und Delirmanagement in der Intensivmedizin. S3-LL (DGAI/DIVI) (Evidence and consensus-based German guidelines for the management of analgesia, sedation and delirium in intensive care)—AWMF (DED)—Association of Scientific Medical Societies; 2009 (valid until 01.12.2014) (G-I-N Library; not accessible)
Synopsis of the National Institute for Health and Clinical Excellence guideline for prevention of delirium67	O'Mahony et al	2011	Medline, CINAHL, EMBASE	Delirium: diagnosis, prevention and management. NICE Clinical Guideline 103; 201038
Diagnosis, prevention, and management of delirium: summary of NICE guidance17	Young et al	2010	Medline, EMBASE	Delirium: diagnosis, prevention and management. NICE Clinical Guideline 103; 201038
Evidence-based guideline: Acute confusion/delirium71	Sendelbach et al	2009	Medline	Acute Confusion/Delirium; 200950
Clinical practice guidelines for the management of delirium in older people in Australia72	Tropea et al	2008	Medline, CINAHL, PsycINFO	Clinical practice guidelines for the management of delirium in older people; 200656

NICE, National Institute for Health and Care Excellence.

Three dissemination papers were published in 2003–2007,^73–75 and two before 2003.^{76 77} Six of the 11 retrieved guidelines (2008–2013) had guideline summaries published on the NGC website³⁵ (see table 5).

Table 5.

National Guideline Clearinghouse (NGC) published summaries for delirium guidelines (2008–2013)

NGC guideline summary name	Guideline developer	Year	Delirium clinical practice guidelines referred to	Country
Guideline summary NGC-9692	Society of Critical Care Medicine	2013 (revision of 1995 version)	Clinical practice guidelines for the management of pain, agitation and delirium in adult patients in the intensive care unit45	USA
Guideline summary NGC-9461	Family Practice Oncology Network; Medical Services Commission	2011	British Columbia guidelines: Palliative care for the patient with incurable cancer or advanced disease. Part 2: Pain and symptom management54	Canada
Guideline summary NGC-8072	National Clinical Guideline Centre for Acute and Chronic Conditions	2010	Delirium: diagnosis, prevention and management. NICE Clinical Guideline 10338	UK
Guideline summary NGC-8381	Registered Nurses’ Association of Ontario	2004 (addendum released May 2010)	Caregiving strategies for older adults with delirium, dementia and depression46	Canada
Guideline summary NGC-8380	Registered Nurses’ Association of Ontario	2003 (addendum released May 2010)	Screening for delirium, dementia and depression in older adults48	Canada
Guideline summary NGC-7208	University of Iowa College of Nursing	2009 (revision of 1998 version)	Acute Confusion/Delirium50	USA

NICE, National Institute for Health and Care Excellence.

From the literature search, only three studies (all cited in EMBASE) described the evaluation of implemented delirium guidelines. Mudge et al⁷⁸ reported an Australian controlled trial of implementation of a 2006 guideline on the management of delirium in older people on a medical ward compared with a control ward.⁵⁶ Twenty-two per cent of the patients were delirious on admission and no incident cases of delirium were identified. In the delirious subgroup, there was a trend to a lower use of antipsychotic medications (26% vs 41%, p=0.46). In the intervention group there was a longer duration of admission, a trend to reduced inpatient mortality, and significantly fewer patients were discharged with persistent delirium (32% vs 71%, p=0.016). The authors commented that implementation was costly, aside from the cost due to increased length of acute ward stay. Voellinger et al⁷⁹ described the effect of guideline implementation in a general hospital in Switzerland on nurses' and physicians' knowledge after a 1-hour education session using pre-MCQ and post-MCQ questionnaires. The study illustrated the effort required for implementation but no assessment was made of clinical outcomes. In 2003, Young and George reported a pre-evaluation and postevaluation of the implementation using cluster randomisation of a British Geriatrics Society delirium guideline implemented in five hospitals with graduated ‘intensity’.⁸⁰ One hospital received the ‘low-intensity’ intervention: this consisted of feedback of baseline data alone from an earlier study. From chart review of patients with a diagnosis of delirium, the baseline data included frequency of mental test score administration, alcohol history, use of non-pharmacological management strategies and sedation, in addition to complications and length of stay. Two hospitals received the addition of guideline distribution to nurses and doctors (‘medium-intensity’ intervention); two hospitals received additional teaching sessions (‘high-intensity’ intervention). There was an improvement in process and outcome of care in the high-intervention group only, but this was not statistically significant. The authors concluded that guidelines themselves did not improve delirium management.

Discussion

This study describes the systematic search process and quality appraisal of published delirium clinical guidelines across clinical settings. The key findings from the literature search were the importance of the grey literature search strategy to source delirium guidelines, as opposed to sourcing guideline summary papers and evaluation studies in indexed databases, and the lack of currency for many guidelines. Of note, nine of the 11 guidelines published between 2008 and 2013 were found solely through the grey literature search. This is in keeping with our previous experience of sourcing delirium guidelines, where a lower proportion of guidelines were retrieved from biomedical electronic databases.³² Similarly in a search for dementia guidelines, Azermai et al⁸¹ retrieved over half of the appraised guidelines from the web. For our study, the NGC website was an effective way to identify and source recently published guidelines.^{35 82} Only the CCSMH guideline for older adults with delirium at the end of life did not have an NGC guideline summary.⁴⁷ However, the six delirium guidelines with NGC summaries varied with their final AGREE II ‘Rigour of Development’ domain ratings, with two rating below 40% in this domain.^{50 54} In contrast to sourcing delirium guidelines, bibliographic databases appear to be the best source for guideline summary papers as part of a guideline dissemination strategy. In this study, all the summary papers of delirium guidelines published between 2008 and 2013 were cited in the Medline database, as well as some of the other commonly used electronic databases. It was disappointing to find little evidence for further knowledge translation with only three publications from 2003 to 2013 describing the formal evaluation of fully implemented delirium guidelines (as opposed to published reports of the implementation process and facilitators and barriers to guideline implementation) being retrieved. Almost 50% of the delirium guidelines were >5 years old, yet the median lifespan of NICE guidelines is 60 months.⁸³ It is important for guidelines to maintain currency and remain relevant and valid with mechanisms designed to incorporate important new research findings efficiently. A ‘red flag monitoring system’ has been proposed to allow timely incorporation of exceptional updates with the collation of alerts and medical product safety information from national authorities, for example, US Food and Drug Administration (FDA), Health Canada, European Medicines Agency.⁸⁴ Partial updating of guidelines from systematic monitoring may be more practical, cost-effective and provide more timely recommendations than using scheduled full updates (usually every 3–5 years).

A key finding of the quality appraisal was the variation in quality of formal delirium clinical guidelines across all six AGREE II domains and lack of practical implementation and audit tools for many guidelines. Performing an initial AGREE II assessment with two appraisers evaluating the ‘Rigour’ dimension was a practical way to reduce the number of delirium guidelines requiring full AGREE II appraisal. The 2010 NICE delirium guideline received the highest AGREE II overall quality rating and domain scores for all domains, except ‘Editorial Independence’.³⁸ Considering all seven delirium guidelines (published between 2008 and 2013) selected for the full AGREE II appraisal, the ‘Applicability’ and ‘Editorial Independence’ domains were frequently the lowest rated. This result is similar to findings by other authors appraising guidelines with the original AGREE instrument or updated AGREE II version.^{81 85–87} As the four items for the AGREE II ‘Applicability’ domain (facilitators and barriers to guideline application, provision of implementation tools and/or advice on how to put recommendations into practice, consideration of resource implications, and provision of monitoring and/or audit criteria) help healthcare providers operationalise a guideline into clinical practice as well as monitor its effectiveness, a low-rating ‘Applicability’ domain may lead to challenges with sustainability of the guideline once implemented. As part of the AGREE II ‘Editorial Independence’ domain, it is vital that members of a guideline development committee have declared any potential conflicts of interest, especially financial, or links with industry, and that this information is clearly accessible within the guideline itself or as an appendix, to ensure that the reader is well informed and can assess for potential bias.²⁹

A strength of this study is the usage of an experienced university health sciences librarian to assist with the development of the search strategy for the systematic literature search. Adding to the methodological rigour and reliability of the AGREE II assessment, a complement of four (rather than two) appraisers conducted the full AGREE II quality assessment of the delirium guidelines. This is in keeping with the recommendations of the AGREE II user's manual.³³ Although the appraisers were inexperienced in guideline evaluation, all had completed the AGREE II online training module. All appraisers had clinical and research interests in delirium in palliative care or older populations. A limitation is that it was not possible to retrieve the full text for all the citations from the literature search or from the G-I-N International Guideline Library, in addition to the language restriction to English and French for the second-level screening. While it is necessary to be a subscribed G-I-N member to retrieve guidelines from the G-I-N website, it is possible to search the International Guideline Library for identification of guidelines and whether the guideline status is ‘in development’, ‘published’ or ‘under review’.³⁶ For this study, the full AGREE appraisal focused on ‘recent’ delirium guidelines that had been published within the previous 5 years at the time of the literature search. The literature continues to evolve with the subsequent development or revision of guidelines, which were not included as part of this study. The CCSMH has published a 2014 update (in English and French) for its 2006 guideline on ‘The assessment and treatment of delirium’ in seniors.⁸⁸ From the G-I-N website,³⁶ the Dutch College of General Practitioners published Delier (M77) (Delirium) on 1 April 2014. This appears to be an update of a 2003 (M77) guideline. From the NICE website, the next review date for the NICE CG103 guideline is 2018, as there was found to be no requirement to update this guideline at the last review in January 2015.³⁸

This study confirms the importance of systematic grey literature searches to source tangible delirium clinical practice guidelines, and to ensure important guidelines are not overlooked. Poor accessibility of a guideline reduces compliance, in addition to its complexity and length.⁸⁹ This study also illustrates the importance of using a formal appraisal tool, such as a full AGREE II assessment, to ensure that only high-quality delirium guidelines are adapted by clinicians for use in practice. Strengthening of items derived from the AGREE II ‘Applicability’ domain would improve many of the delirium guidelines that we reviewed. By providing a robust ‘Applicability domain’, guideline developers can assist busy healthcare providers at the bedside with specific audit recommendations and provision of user-friendly templates for comparative audits preguideline and postguideline implementation, in addition to practical implementation tools. We envision that by using the recently published AGREE Reporting Checklist (available on the AGREE Enterprise website as a free and open access resource (http://www.agreetrust.org)), guideline development groups will improve the comprehensiveness and completeness of new guidelines across all AGREE II domains.⁹⁰ The AGREE II instrument was developed from the original AGREE version.^{91 92} While to the authors' knowledge, the AGREE II has not been subsequently revalidated, AGREE II is widely accepted and has been used extensively to appraise guidelines.⁹³ The AGREE II does not formally evaluate the content of a guideline, including guideline criteria for evidence or the validity of guideline recommendations. Indeed it has been shown that the methodological quality of guidelines should not infer content validity.⁹⁴

We found limited evidence of evaluation studies of delirium guidelines (with or without local adaptation) in their implemented settings. In addition to assessing the impact of guidelines on the quality of care, other identified desired outcomes, and assessment of guideline sustainability and cost-effectiveness, there should also be assessments for unintended consequences or undesirable outcomes to ensure that patient health and family outcomes are improved. A limitation to this study in assessing impact of guidelines is that it only included scientific literature and guidelines that had been published or retrieved by the grey literature search. The extensive quality assessment and improvement initiatives that are in place in individual institutions and networks across the world and their impact on care were not examined. Considering that the development and implementation of formal guidelines can be challenging, taking time, money and resources at the local level, a possible practical starting point to improve care is to encourage uptake and benchmarking of quality improvement measures related to delirium. NICE quality standards are concise and measurable enabling the development of local quality measures.⁹⁵ For example, the NICE Quality Standard 63 for delirium management recommends five key standards.⁹⁶

Future research needs to ensure that clinical practice guidelines are of high quality, user-friendly, relevant and non-biased as only ‘rigorous’ guidelines should be adapted into formats suitable for use in daily clinical practice. Crucial outstanding factors to enable guidelines to change practice are effective interprofessional implementation strategies at the local level to optimise uptake, maximum accessibility of guideline key points with practical and simple interprofessional clinical prompts and algorithms available at the point of care, and ensuring ongoing sustainability with long-term organisational support. A challenge for delirium guidelines in general remains the need for more primary research evidence (and higher quality studies) to inform them.