Development of the Ketamine Side Effect Tool (KSET)

Abstract

Background:

Currently, no specific, systematic assessment tool for the monitoring and reporting of ketamine-related side effects exists. Our aim was to develop a comprehensive Ketamine Side Effect Tool (KSET) to capture acute and longer-term side effects associated with repeated ketamine treatments.

Methods:

Informed by systematic review data and clinical research, we drafted a list of the most commonly reported side effects. Face and content validation were obtained via feedback from collaborators with expertise in psychiatry and anaesthetics, clinical trial piloting and a modified Delphi Technique involving ten international experts.

Results:

The final version consisted of four forms that collect information at time points: screening, baseline, immediately after a single treatment, and longer-term follow-up. Instructions were developed to guide users and promote consistent utilisation.

Limitations:

Further evaluation of feasibility, construct validity and reliability is required, and is planned across multiple international sites.

Conclusions:

The structured Ketamine Side Effect Tool (KSET) was developed, with confirmation of content and face validity via a Delphi consensus process. This tool is timely, given the paucity of data regarding ketamine’s safety, tolerability and abuse potential over the longer term, and its recent adoption internationally as a clinical treatment for depression. Although based on data from depression studies, the KSET has potential applicability for ketamine (or derivatives) used in other medical disorders, including chronic pain. We recommend its utilisation for both research and clinical scenarios, including data registries.

1. Introduction

Ketamine is classified as an N-methyl-D-aspartate (NMDA) receptor antagonist, although its pharmacological profile is complex and its affinity for numerous receptors has been identified (Mathew et al., 2012). Numerous articles including original studies, narrative reviews and meta-analyses have been published, endorsing the short term efficacy of ketamine in depression (McGirr et al., 2015). Only a minority of these studies, however, have systematically assessed ketamine’s safety, tolerability and abuse potential. Very few have examined the safety (or efficacy) of repeated treatments and longer-term use (Schoevers et al., 2016), though repeated treatments are increasingly being used in randomised controlled trials (RCTs), open label studies, case studies and some clinical services (Short et al., 2018).

Ketamine is now increasingly used off-label to treat a number of medical conditions, including depression. Recently, the FDA has granted approval of the S-enantiomer of ketamine, esketamine, for patients with treatment-resistant depression (Kim et al., 2019). Its distribution and prescription are subject to continuing monitoring to mitigate the risks of misuse, abuse and serious adverse outcomes from dissociation, sedation, and blood pressure changes (U.S Food and Drug Administration, 2019).

Serious safety concerns have been reported in other population groups exposed to repeated use of ketamine, such as patients with chronic pain and recreational drug users. In a review conducted by the World Health Organization (WHO), urinary tract symptoms were documented as a regularly reported side effect of ketamine, and liver toxicity, cognitive changes and dependence as potential harms (World Health Organization, 2015).

In particular, the acute cognitive effects of ketamine treatment for depression have been minimally studied. In healthy participants, acute infusions using larger ketamine doses have been associated with significant short-term deficits particularly in working memory, source memory and episodic memory, as well as subjective cognitive effects (e.g., impaired memory, confusion) (Morgan et al., 2004a, 2004b).

Only a few studies have examined possible cumulative cognitive effects of repeated ketamine treatment in depressed patients (e.g., (Blier et al., 2012; Diamond et al., 2014; Galvez et al., 2018; George et al., 2017; Irwin et al., 2013; Shiroma et al., 2014). Cognitive deficits following repeated use of NMDA antagonists have been reported in animal studies (Jentsch et al., 1997; Mandillo et al., 2003) and in recreational users of ketamine (George et al., 2017). In humans, these deficits have been observed primarily on measures of verbal fluency, episodic memory and attention/working memory (Morgan et al., 2004b; Morgan et al., 2010). Despite the current failure to detect any negative effects with repeated treatment in depression research trials, the potential for more chronic cognitive side effects with higher frequency, doses or number of treatments cannot be ruled out.

In a recent systematic review (Short et al., 2018) on the use of ketamine to treat depression, we reported that the majority of depressed patients receiving ketamine, particularly via intravenous methods, experienced acute side effects, the most common being headache, dizziness, dissociation, elevated blood pressure and blurred vision. Conclusions could only be made regarding single dosing and acute side effects as a lack of data was available regarding the side effects of repeated dosing and possible cumulative and longer-term risks. A number of nonspecific outcome measures were used to collect safety and tolerability data immediately and up to four hours after a single treatment, but these varied in the type and definition of adverse effects covered. Most of the monitoring was conducted only over the short term (i.e. for a few hours after the treatment), and if completed, was predominantly reported in ad-hoc form rather than using structured instruments. Overall, many studies relied on passive surveillance for the majority of potential side effects (i.e. clinical observation and/or spontaneous reporting) rather than active inquiry (Short et al., 2018).

To provide a robust evidence base, it is important that clinical studies of ketamine collect and report data about side effects in a systematic way to enable a true and fair comparison of its propensity for both common and potentially serious adverse effects, both acutely (in the hours after treatment) and on a cumulative (repeated doses) and longer term basis. Studies that rely solely on passive surveillance to collect safety and tolerability data are likely to underestimate the prevalence of subjective, intimate or other undisclosed adverse effects because patients appear to under-report treatment-related symptoms when asked general questions about tolerability (Pope et al., 2010).

To date, no specific, systematic assessment tool for the monitoring and reporting of ketamine-related side effects has been developed. The purpose of this paper is to describe the development of the KSET, a clinically feasible approach, relying on easy-to-use checklists, that can be used both in research trials and routine clinical practice to monitor for a range of common ketamine side effects, both acute (i.e. after a single treatment) and cumulative (repeated doses). When administered together with a physical examination, cognitive testing and biochemical tests, it can form part of a more comprehensive assessment regarding ketamine safety, tolerability and potential risks over time.

2. Methods

Development of the KSET consisted of three phases: 1/a systematic literature review to identify the most commonly reported side effects associated with ketamine use (Short et al., 2018) 2/development of a draft set of Ketamine Side Effect Tool (KSET) questionnaires that were piloted in a clinical trial, and 3/face and content validation, including revision of the KSET questionnaires based on feedback from collaborators and through an iterative process utilising a modified Delphi Technique (Eubank et al., 2016).

2.1. Systematic literature review

A systematic literature search was conducted to identify the most commonly reported side effects related to ketamine use in depression. MEDLINE, PubMed, PsychINFO and Cochrane Database articles between 1 January 1999 to 30 December 2016 were searched to identify all papers with the following terms: ketamine* AND (depress* OR affective* OR mood* OR bipolar*) AND (safe* OR side* OR adverse*). Filters were applied to limit the results to human studies and adult populations (defined as aged 18 or more years). No language restrictions were imposed. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. (Moher et al., 2009) For the full details on the systematic review methodology, see Short et al. (2018).

2.2. Development of the KSET questionnaires

Items for the questionnaires were generated by extracting the most commonly reported side effects experienced by study participants versus placebo from the systematic review results. This included review of 60 studies and 899 patients who had received at least one dose of ketamine. These preliminary side effect items were incorporated into the safety evaluations for the Ketamine for Adult Depression Study (KADS; ANZCTR registration number ACTRN12616001096448), to complement other commonly used side effect scales (e.g., Clinician Administered Dissociative States Scale; CADSS). KADS is a multi-site clinical trial with locations in Australia and New Zealand involving ketamine administration to depressed participants. The study was approved by the Sydney Local Health District Human Research Ethics Committee and commenced in mid-2016 with trial locations based in New South Wales, Victoria, South Australia, Queensland and New Zealand.

The initial KSET forms were subsequently developed based on the items piloted in KADS as a basis, with optimisations to content and form design/structure informed by the experiences of trial staff.

2.3. Face and content validation

Feedback about face and content validity were sought throughout the development process to ensure that the KSET would appropriately capture side effects of ketamine treatment observed in practice in a useful manner. We first sought general comments on the initial KSET forms from several collaborators involved in KADS with expertise in the fields of psychiatry, anaesthetics and neuropsychology. This feedback was requested in an open-ended and unstructured manner and the comments provided by respondents led to further refinements to content and structure.

We then solicited formal, structured feedback regarding face and content validity from international researchers and clinicians experienced in the use of ketamine therapy for depression or other disorders via a modified Delphi technique (Dalkey N, 1963). The Delphi technique is recommended for use in the healthcare setting as a reliable means of determining expert group consensus where there is little or no definitive evidence, and where opinion is important. This method is an iterative process that uses a systematic progression of repeated rounds of voting ideally with a diverse panel of experts (Eubank et al., 2016; Meshkat et al., 2014). The modified Delphi technique employed in this study consisted of closed, concise revision between two authors (B.S and V.D), followed by subsequent Delphi rounds. This included the following steps:

1. Experts were identified based on their experience with ketamine for depression either clinically and/or in research. We aimed to include clinicians/researchers with expertise in psychiatry from international backgrounds, where possible.

2. Experts were individually invited to review the forms and instructions and provide feedback regarding: (a) the relevance and appropriateness of side effect symptoms in each form, (b) their feasibility/usefulness/comprehensibility in each form, and (c) recommendations for cognitive testing and clinical investigations provided in the forms/instructions. Feedback regarding these areas was requested via email in the format of a structured questionnaire, but open comments were also solicited. Experts were allowed up to 2 months to provide their first round of feedback.

3. BS and VD de-identified and collated the feedback before conducting a preliminary review to identify repeated comments (i.e., when multiple experts make comments of the same nature) and discuss whether each comment should or should not be endorsed. The results of this preliminary review were then presented to the research team (including VV, CL, VG, DM and AB) to decide whether each aspect of the KSET forms should be retained as-is, revised, or removed. Feedback was automatically endorsed if more than three experts had commented unanimously on the same issue. Decisions for all other comments and feedback were made based on discussion and agreement within the research team.

4. Experts were invited to review the revised questionnaires/instructions and provide a second round of feedback via email. General comments were requested in an open-ended manner, but more specific feedback was also requested regarding: (a) the appropriateness of the side effect symptoms included, (b) whether the forms were conducive to patient self-rating, (c) the practicality and functionality of the questionnaires, and (d) the usefulness of the instructions. Experts were allowed approximately 6 weeks to provide their second round of comments.

5. The review and revision process described in Step 3 were repeated by the research team with the aim of finalising the KSET.

Once revisions resulting from the modified Delphi analysis were complete, the questionnaires (including instructions) were presented to a colleague unfamiliar with the KSET for their review (to provide a neoteric and external perspective), and road tested in some patients being treated clinically by the team. Some further clarifications to phrasing and refinements in formatting were incorporated into the final version of the KSET based on these suggestions and experiences. The finalised questionnaires and instructions were then circulated to all participants of the Delphi process for their approval. Further comments were not sought at this stage, but typographical, grammatical and formatting errors were corrected if brought to the awareness of the research team.

3. Results

3.1. KSET development and validation

The most commonly reported side effects derived from the systematic review are presented and described in Short et al. (2018). Because of a lack of data regarding some specific side effects captured by the systematic review, further literature searches and consultation were required regarding urinary and cognitive side effects, as well as certain items included on the screening form, including ketamine use in pregnancy and pre-existing glaucoma.

The main considerations incorporated into the initial build of the KSET following the experiences of KADS trial participants and staff were as follows:

• refining the phrasing of individual side effect items to minimise potential ambiguity of interpretation

• changing answer options from a tick/box system to a numerical (Likert-like) approach

• adding a measure of baseline symptomatology, to assist with comparison and interpretation of symptoms at follow-up intervals

• including an overall tolerability rating for the acute treatment questionnaire, to help guide dosing in subsequent treatment sessions.

To increase the efficiency of completing the questionnaire, the side effects were initially classified into ‘body categories’ (e.g., neurological, gastrointestinal, psychotomimetic) based on the systematic review findings and categories used in existing questionnaires for reporting adverse drug reactions (Jarernsiripornkul et al., 2002), with an option to report other side effects not covered within the category sections. However, initial feedback from the KADS collaborators revealed difficulties utilising the body category classification system for some of the symptoms due to overlap across multiple categories (e.g. hallucinations could be categorised in both psychotomimetic and psychiatric categories), which led to the subsequent removal of the body categories.

Overall, four collaborators in KADS provided general comments regarding the initial KSET that was developed. Apart from the de-categorisation of side effects, the main changes which resulted from this feedback were as follows:

• adding more detailed information regarding the recommended timing of cardiovascular monitoring and discharge assessments in the acute treatment form

• providing more detailed instructions about how to complete and use the forms

• separating baseline and follow-up measures of long-term, cumulative side effects into two separate forms to promote clarity

• adding items regarding potential cumulative side effects based on literature and observations related to ketamine used for chronic pain patients and recreational users of ketamine. These included ‘abdominal pain/cramps’, ‘skin changes’ and urinary tract symptoms.

• recommending a two-step approach for the assessment of longer-term urinary and cognitive side effects (see Table 1), which recommends that objective tests/investigations be completed if a patient subjectively reports problems with urination or memory.

Table 1.

Individual cognitive tests proposed for safety monitoring for repeated ketamine treatment (i.e., choose one measure for each domain).

Verbal episodic memory	Working memory/Attention	Verbal Fluency
Rey Auditory Verbal Learning Test (RAVLT)	Symbol Digit Substitution Test (SDMT)	Controlled Oral Word Association Test (COWAT)
Hopkins Verbal Learning Test – Revised (HVLT-R)	Conners Continuous Performance Test 3rd Ed. (CPT 3)	Delis-Kaplan Executive Function System (D-KEFS) Verbal Fluency subtest
California Verbal Learning Test – Second Ed. (CVLT-II)

RAVLT (Rey, 1964), HVLT-R (Benedict et al., 1998), CVLT-II (Delis et al., 2000), SDMT (Smith, 1991), CPT 3 (Conners, 2000), COWAT (Benton and Hamsher, 1989), D-KEFS (Delis et al., 2001).

Twelve experts were invited to participate in the modified Delphi process. Of the 12 invited, all 12 participated in the first round of review, nine responded with a second round of feedback and ten responded to the third and final round. Feedback from across the two rounds of feedback led to refinements to the language used for symptom questioning, as well as changes to the structure and organisation of the forms to improve clinical utility and efficiency. Key amendments resulting from the feedback provided by Delphi experts included:

• adding a space to record details of the ketamine dose, route, and time of administration, to facilitate interpretation of the side effects reported

• collecting more details regarding a patient’s previous experience(s) with ketamine, to provide clearer picture of the patient’s risk of side effects, including risk of dependence

• including instructions for use as part of the forms, rather than in a separate manual

• simplifying the way in which cognitive test and clinical investigation results are recorded, to minimise clinician burden

• providing anchor points/definitions for severity, to promote consistency when rating and interpreting the severity of the symptoms reported

• adding a section for clinicians to note any ketamine administration difficulties or effects, i.e., nose bleed if given intranasally

• adding an area for a medical record number to be entered

3.2. KSET: final version (Appendix 1)

The final version of the KSET is designed to be a clinical tool that systematically captures side effect information and assists with clinical decision-making. To cover all time periods for potential side effect development relevant to a course of ketamine treatment, the KSET is divided across four separate forms: Screening, Baseline, Acute Treatment, and Follow-Up. Together, these forms aim to collect information about (a) screening and possible caution or contraindications, (b) baseline symptoms, (c) acute side effects experienced immediately after dosing, (d) side effects that have emerged since the preceding treatment dose (which may represent potential cumulative or delayed side effects), and finally (e) whether side effects are experienced after the treatment course or over longer term intervals. The Acute Treatment form also includes space to record vital signs and assessments for discharge to assist with monitoring during treatment sessions, with all sections colour coded according to time of measurement (see Fig. 1 and Appendix 1).

Fig. 1.

KSET forms, time points and aims.

Each form contains an instruction page to provide guidance regarding the purpose of the form, how it should be completed and other relevant clarifications or recommendations. The questions are designed to make it possible for the patient to fill in some side effect information on the forms directly, for subsequent review by the clinician. All forms collect information about the patient, clinician, and assessment date/time.

4. Discussion

Despite growing interest in ketamine as an antidepressant and its rapid treatment effects, to-date potential side effects have not been adequately explored after repeated dosing and over the longer term. All pharmacological treatments can cause unwanted side effects. As such, systematic assessment and monitoring of anticipated side effects is a vital part of good research and clinical practice. Up to 10% of hospital admissions are due to adverse drug reactions (National Patient Safety Agency, 2007) and according to National Health Service (NHS) data, most of these are preventable.(Howard et al., 2007; Pirmohamed et al., 2004) As treatment with ketamine and its derivatives will likely involve multiple and repeated doses over an extended time period, it is crucial to determine whether the potential side effects outweigh the benefits to ensure it is safe for this purpose.

Subsequently, side effects should be actively monitored, using standardised structured questionnaires. Side effect questionnaires can be open-ended or checklist based. Compared to open-ended questionnaires, checklist-based questionnaires are more sensitive in identifying potential adverse effects (Bent et al., 2006; Sheftell et al., 2004).

Our aim was to develop a tool that actively and systematically enquires about ketamine side effects over the short- and long-term. The KSET is unique in that it covers the most common potential side effects of ketamine in one tool (other than cognitive functioning) and further, recommends inquiry about ketamine-related side effects and tolerability over different time periods, including the longer term, which has not been adequately addressed in the literature to date. It also includes a screening form, to assess the risk of administering ketamine to patients with pre-existing or co-morbid medical conditions, such as those with a history of high blood pressure or heart disease.

Prior to its development, researchers and clinicians relied on multiple questionnaires to assess for potential side effects, including the Brief Psychiatric Rating Scale (BPRS), the Clinician Administered Dissociative States Scale (CADSS), the Young Mania Rating Scale (YMRS), the Hamilton Anxiety Rating Scale (HAM-A) and the Systematic Assessment for Treatment Emergent Effects (SAFTEE), (Short et al., 2018) to name a few. Ultimately, having one standardised tool will assist the field in identifying any safety or dependence problems with longer term use and narrow down what dose, frequency, route and durations of ketamine treatment work best.

Although the KSET has been developed from a systematic review of side effects experienced in patients with depression, there is no evidence that these are exclusive to this disorder. Subsequently, it is likely the KSET could be applicable to other disorders in which ketamine or ketamine derivatives are used as treatment, such as chronic pain disorders, or for evaluating the adverse effects recreational users may experience from ketamine misuse or dependence.

5. Limitations

We aimed for the KSET questionnaire to be comprehensive and to include the most common side effects reported in the literature to date. Given this, publication bias may be present in that articles that describe significant results are usually published over those that do not. Attempts to minimise this were made by also searching grey literature, white papers and including conference abstracts if they were not published as part of another research article elsewhere.

Further testing of the KSET, including additional validity and reliability evaluations, are recommended. In terms of validity testing, content and face validity have been considered. Other validities were more difficult to assess, including criterion validity and construct validity. Criterion validity indicates the effectiveness of a questionnaire in measuring what it purports to measure. The responses on the questionnaire being developed are checked against an external criterion, or gold standard, which is a direct and independent measure of what the new questionnaire is designed to measure (Kirshner and Guyatt, 1985). Because no other comprehensive side effect questionnaire has been developed specific to ketamine, this was not possible. An approach could be to assess how the KSET compares against other measures that have been used, such as the BPRS, CADSS or SAFTEE.

6. Conclusion

Systematic assessment of anticipated side effects is recommended as part of good clinical care. To our knowledge, the KSET is the only structured tool designed to evaluate side effects specifically associated with ketamine use, in both acute and longer-term time frames. Although based on data from depression studies, it is likely the KSET has potential applicability for other medical disorders, including chronic pain. We recommend its use for both research and clinical scenarios, as well as for data collection and side effect registries.