The Withdrawal Assessment Tool - Version 1 (WAT-1)

Abstract

Objective

To develop and test the validity and reliability of the Withdrawal Assessment Tool - Version 1 (WAT-1) for monitoring opioid and benzodiazepine withdrawal symptoms in pediatric patients.

Design

Prospective psychometric evaluation. Pediatric critical care nurses assessed eligible at-risk pediatric patients for the presence of 19 withdrawal symptoms and rated the patient’s overall withdrawal intensity using a numeric rating scale (NRS) where 0 indicated no withdrawal and 10 indicated worst possible withdrawal. The 19 symptoms were derived from the Opioid and Benzodiazepine Withdrawal Score (OBWS), the literature and expert opinion. Setting: Two Pediatric Intensive Care Units (PICU) in university-affiliated academic children’s hospitals.

Patients

83 pediatric patients, median age 35 months (IQR: 7months -10 years), recovering from acute respiratory failure who were weaning from more than 5 days of continuous infusion or round-the-clock opioid and benzodiazepine administration.

Interventions

Repeated observations during analgesia and sedative weaning. A total of 1040 withdrawal symptom assessments were completed, with a median (IQR) of 11 (6-16) per patient over 6.6 (4.8-11) days.

Measurements and Main Results

Generalized linear modeling was used to analyze each symptom in relation to withdrawal intensity ratings, adjusted for site, subject and age group. Symptoms with high redundancy or low levels of association with withdrawal intensity ratings were dropped, resulting in an 11-item (12-point) scale. Concurrent validity was indicated by high sensitivity (.872) and specificity (.880) (WAT-1 ≥3 predicting NRS ≥4). Construct validity was supported by significant differences in drug exposure, length of treatment and weaning from sedation, length of mechanical ventilation and intensive care unit stay for patients with WAT-1 scores ≥3 compared to those with lower scores.

Conclusions

The WAT-1 shows excellent preliminary psychometric performance when used to assess clinically important withdrawal symptoms in the PICU setting. Further psychometric evaluation in diverse at-risk groups is needed.

Introduction

Patients in the pediatric intensive care unit (PICU) frequently receive prolonged analgesia and sedation to provide pain relief and blunt physiological stress responses.^1-3 Iatrogenic withdrawal occurs when these drugs are then stopped abruptly or weaned too rapidly, causing central nervous system hyperirritability, autonomic system dysregulation, gastrointestinal dysfunction and motor abnormalities.^4;5 The risk of withdrawal is influenced by the length and amount of exposure to opioids and benzodiazepines, rising to over 50% after 5 days of continuous infusion or around-the-clock administration.⁶ Iatrogenic withdrawal can complicate medical treatment, cause distress to patients and families and prolong recovery and hospital stay.^1;4;7 Although the prevalence of iatrogenic withdrawal is unknown, one survey found it was a problem in 94% of PICUs. ⁸

Despite the fact that iatrogenic withdrawal has been observed for over 20 years in infants and children receiving intensive care, a gold standard for the diagnosis or a measure of withdrawal symptom intensity does not exist. Without a measurement tool, early diagnosis and systematic evaluation of the effectiveness of treatment is difficult.^2;5;6;9

Current clinical assessment tools are derivatives of the Finnegan Neonatal Abstinence Score (NAS) developed in 1975 for the assessment of withdrawal symptoms in otherwise healthy neonates with prenatal drug exposure.¹⁰ Our previous investigation of the Opioid and Benzodiazepine Withdrawal Score (OBWS) provided preliminary validity and reliability of a withdrawal assessment measure for use with infants and young children in intensive care settings. ^6;11 When tested, we found that sensitivity and specificity of the OBWS were only moderate compared to nurses’ clinical judgment of withdrawal intensity and we identified several usability difficulties, including the need for further training and improved inter-rater reliability, confusion over when to assess the patient and the burden of the frequency of assessment.

The Neonatal Withdrawal Index (NWI) ¹² uses a different approach and has demonstrated improved diagnostic accuracy for detecting withdrawal in neonates with perinatal drug exposure. The NWI is performed only twice per day and involves assessment of a brief list of key withdrawal signs and symptoms before, during and after a routine standard physical assessment and caregiving procedure. This has advantages over previous methods of four-hourly observation of the patient in a ‘resting’ state in terms of ease of use and efficiency. Moreover, it minimizes variability associated with difficulty in defining a resting state, standardizes the observation, and reduces the bias that occurs with frequent serial measurement. We hypothesized that the procedures and structure of the NWI could provide the basis for a new method of iatrogenic withdrawal assessment that had improved reliability, validity, and ease of use in diagnosing withdrawal in older infants and children receiving intensive care. We describe here the development and preliminary psychometric evaluation of the new Withdrawal Assessment Tool - Version 1 (WAT-1) for use with critically ill children who are exposed to opioids and benzodiazepines for prolonged periods.

Material and Methods

Design

A multicenter prospective repeated measures study was conducted to evaluate and refine an instrument developed to assess iatrogenic withdrawal symptoms in pediatric critical care patients. Psychometric evaluation of the new measure included examining response distributions overall and by age, inter-item redundancy, factor structure, and construct validity by comparing scores across groups that were expected to differ (known-groups validity), and analyzing the association of scores with other clinical variables (e.g., amount of drug exposure and length of weaning) hypothesized to be indicative of withdrawal severity (concurrent and predictive validity).

Patient Enrollment

The study was conducted within the context of a clinical trial testing a sedation management protocol in pediatric patients (2 weeks to 18 years of age) supported on mechanical ventilation for acute respiratory failure in the PICU of two children’s hospitals (Children’s Hospital National Medical Center, Washington, DC; Children’s Hospital of Wisconsin, Milwaukee, WI).¹³ Both PICUs are large (22-24 bed) university-affiliated regional referral centers. Neither site had a structured sedation weaning policy in place. The institutional review boards at both sites approved the study. Consent for data collection was obtained from all parents/guardians. Data were collected from February 2004 to April 2006.

All patients exposed to greater than 5 days of continuous infusion or regular around-the-clock dosing of opioids were assessed for withdrawal symptoms twice daily at 8 am and 8 pm (and at other times if clinically indicated) from the day that opioid weaning started until 72 hours after the last opioid dose.¹ The morning and the highest other score were used in analyses. Patients exited the study at PICU discharge or after 28 days.

Instrument Description

The 19 symptoms of opioid and benzodiazepine withdrawal were derived from the OBWS, documented in the withdrawal literature^1;4 and supported by our previous study of withdrawal symptoms in the PICU setting.¹¹ The 19-item assessment consisted of (1) a review of the patient’s record for the past 12 hours, (2) direct observation of the patient for 2 minutes, (3) patient assessment during a progressive stimulated¹⁴ exam routinely performed to assess level of consciousness at the beginning of each 12-hour shift, and finally (4) assessment of post-stimulus recovery. The procedure for the standard progressive stimulation has been previously reported¹⁴. During normal cares, the nurse first calls the patient’s name in a calm voice. If the patient does not respond, the nurse calls the patient’s name and gently touches the patient’s arm or leg. If the patient still does not respond, the nurse would assess the patient during a planned noxious procedure, e.g., endotracheal suctioning or repositioning. Patient data assessed from the previous 12 hours included 5 items: any loose or watery stools, any vomiting/wretching/gagging, temperature > 37.8, respiratory rate greater than the child’s baseline, suctioning more than once every two hours. Patient data assessed during the 2-minute pre-stimulus/quiet observation included 5 items: distressed state behavior, any tremor, sweating, uncoordinated or repetitive movements, yawning or sneezing. Patient data assessed during the 1-minute progressive stimulus observation included 8 items: startle to touch, pupils greater than 4mm, increased muscle tone, distressed state behavior; any tremor, sweating, uncoordinated or repetitive movements, yawning or sneezing. The post-stimulus recovery data included 1 item - the time to regain a calm state. Patients received one point for any of the observations. Time to regain a calm state was scored 0 points for 0-2 minutes; 1 point from 2-5 minutes and 2 points for more than 5 minutes.

Bedside nurses received training by the clinical nurse specialist on how to evaluate the presence or absence of each of the 19 symptoms. Training consisted of a didactic review of the data collection instrument followed by completion of a post-test. Inter-rater agreement for withdrawal assessment ratings was established at the start of the study, and re-evaluated every three months by simultaneous scoring by the clinical nurse specialist and the patient’s bedside nurse. There was good inter-rater reliability for the 30 sets of paired ratings (Cohen’s kappa = .80; ICC= .98).

Nurses were also asked to give, based on their clinical experience and judgment, a single overall rating of withdrawal severity on a 0 to 10 numeric rating scale, with zero indicating no withdrawal and 10 indicating the worst possible withdrawal symptoms, Nurses’ clinical judgment of withdrawal intensity is the current standard of care.

Additional Data

Demographic and clinical data included site, age, ethnicity, mortality risk (PRISM III)¹⁵, functional morbidity and cognitive impairment,¹⁶ cumulative and peak daily opioid dosage (morphine equivalents per kilogram of body weight), cumulative and peak daily benzodiazepine dosage (midazolam equivalents per kilogram of body weight) and administration of any other analgesia, sedation or psychoactive medications. Conversion of opioids and benzodiazepines to morphine and midazolam equivalents was performed according to standard methods.¹⁷ Length of mechanical ventilation, length of PICU stay and length of hospital stay, all as number of days, were calculated after discharge or 28 days.

Data Analysis

Descriptive statistics such as mean and standard deviation, or median and interquartile range, were computed for all demographic and clinical variables, and examined as to whether they differed by site. The median (IQR) length of the pre-weaning and weaning phases of opioid and benzodiazepine therapy were computed. The start of the weaning period was defined as the first date of a decrease in daily dosage >10% after 5 consecutive days of opioid treatment. All analyses comparing groups were conducted using chi-square for categorical variables or one-way ANOVA or its non-parametric equivalent, the Mann-Whitney U test, for continuously-distributed variables. Most statistical analyses were performed using SPSS 15 (SPSS Inc, Chicago, IL). For analyses that included multiple observations on the same patient, we used SUDAAN v. 9.0 analysis software (Research Triangle Institute, NC) to account for potential intra-cluster correlation of data within site and within patient.

To evaluate the utility of each withdrawal symptom and identify any that could be dropped, we first examined the inter-item agreement between pre-stimulus and stimulus ratings of the same symptoms (state, tremor, sweating, movement, yawning) using kappa coefficient to determine level of redundancy. A high kappa (>.65) indicated redundancy between the pre-stimulus and stimulus rating of a particular symptom, and only one of the two was chosen to be retained for that symptom. In addition, we examined the prevalence of each symptom across groups that we expected to differ based on nurses’ clinical impression. Using data from all observations, we compared symptom prevalence across the following three groups using the chi-square test in SUDAAN: those with NRS of 0 (no withdrawal), 1-3 (possibly in withdrawal), and 4 or higher (top 20^th percentile of scores, likely in withdrawal). We assessed whether any symptoms had relatively high prevalence among the “no withdrawal” group or did not differentiate the “no withdrawal” and “likely in withdrawal” groups, indicating that these symptoms may not be specific to withdrawal. We also examined the level of missing data for each item to identify those symptoms that may be more difficult to assess.

After dropping redundant, non-specific, or difficult to assess items, exploratory factor analyses were performed on the remaining items using principal components analysis with varimax rotation to examine structural validity of the new Withdrawal Assessment Tool (WAT-1). Based on a scree plot of initial eigenvalues, we examined 3-factor and 4-factor solutions for the total data set containing all assessments, for each age group separately, and for two data subsets created by randomly selecting a single record per patient.

Construct validity of the WAT-1 was evaluated by examining the degree to which peak WAT-1 scores per patient correlated with other indicators of the likelihood of withdrawal, including pre-weaning cumulative opioid and benzodiazepine exposure, analgesia and sedative treatment during weaning, and the duration of weaning. As the data distributions were often skewed for these variables, we examined zero-order correlations among variables by generating Spearman’s rho coefficients in SPSS.

Results

Patient characteristics

Among a total of 245 pediatric patients with acute respiratory failure supported on mechanical ventilation, data were excluded for 26 patients who died and for 4 who did not commence weaning from analgesia or sedation during the 28 day study period. Of the remaining 215 patients, 117 (54.4%) were identified as at risk for withdrawal (>5 days of regular opioid administration), and 83 of them (71%) had withdrawal assessment data for inclusion in the analyses. In 34 cases, patients at-risk for withdrawal had no assessments obtained due to PICU transfer (7 cases) or clinical oversight (26 cases). There were no significant differences in demographic characteristics between the at-risk patients with withdrawal assessments and those without, but assessed patients had greater cumulative opioid (median (IQR): 48.2 (25.5-84.3) vs 7.5 (4.5-18.7) mg/kg; p<.001) and benzodiazepine (median (IQR): 7.6 (4.2-10.4) vs 2.3 (1.0-3.5) mg/kg; p<.001) exposure, longer lengths of mechanical ventilation (median (IQR)=9.9 (6.7-13.8) vs 7.1 (4.6-9.3) hours; p=.004) and PICU stays (median (IQR)=14.0 (10.0-22.5) vs 9.5 (7.0-17.0) days; p=0.015) than non-assessed patients.

Clinical and demographic characteristics of the study patients are shown in Table 1. The demographic characteristics of patients from the two sites (site 1 n=47, site 2 n=36) differed only in race (site 1=62% African American vs. site 2=36%, p<0.05).

Table 1. Demographic and clinical characteristics (n = 83).

Age in months; Median (IQR)	35 (7 - 121)
Age group in years; % 0-2 2-6 Over 6	46 19 35
Female; %	42
Race; % African American Caucasian Other	50 46 4
PCPC > 1; %	33
POPC >1; %	40
PRISM III1; Median (IQR)	9 (6 -16)
Length of mechanical ventilation (days)1,2; Median (IQR)	9.9 (6.7 - 13.8)
Length of PICU stay (days)1,2; Median (IQR)	14 (10 - 23)
Total length of stay (days)1; Median (IQR)	25 (16 - 38)

IQR: Interquartile range;

PCPC: Pediatric Cerebral Performance Category;

POPC: Pediatric Overall Performance Category;

PRISM III: Pediatric Risk of Mortality Score.

Opioid and benzodiazepine weaning

Characteristics of opioid and benzodiazepine exposure and weaning are shown in Table 2. The speed of opioid weaning varied considerably across patients, with one-third characterized by consistent decreases in dosage of variable increments until weaning was completed, another third characterized by multiple decreases and increases in dose during the weaning period, and the final third having single dosage decreases of greater than 25%, some of which were followed by periods where the weaning was slowed or halted.

Table 2. Opioid and benzodiazepine exposure and weaning.

Opioid treatment cumulative dose (mg/kg)1; Median (IQR)	48.2 (25.5-84.3)
Peak opioid dose (mg/kg) 1; Median (IQR)	7.6 (4.2-10.4)
Benzodiazepine treatment cumulative dose (mg/kg) 2; Median (IQR)	24.0 (10.5-62.3)
Peak benzodiazepine dose (mg/kg) 2; Median (IQR)	3.3 (1.9-8.1)
Length of opioid treatment prior to tapering (days); Median (IQR)	6 (5-17)
Length of benzodiazepine treatment prior to tapering (days); Median (IQR)	7 (2-21)
Length of opioid weaning (days from peak); Median (IQR)	11 (1-23)
Length of benzodiazepine weaning (days from peak); Median (IQR)	10 (2-24)

¹Morphine equivalents

²Midazolam equivalents

IQR: Interquartile range

During the weaning period, 32 (39%) patients received 1 to 3 other non-opioid analgesic or sedative drug. However, only 2 drugs, ketamine and chloral hydrate were received by more than 10% of patients. Patients who received these other drugs had greater exposure to opioids (mean peak dose ± SD: 10.4 ± 4.8 vs 6.4 ± 3.9 mg/kg; p=.006; cumulative dose: 77.8 ± 49.3 vs 49.7 ± 40.2 mg/kg; p=.009), but there were no differences in length of the pre-weaning or weaning periods or in benzodiazepine dose or duration compared with patients who were not given other drugs. There were no differences in mean length of opioid or benzodiazepine therapy or weaning and no differences in the mean peak or cumulative opioid doses between the two sites.

Examination of individual withdrawal symptoms

A total of 1040 withdrawal symptom assessments were completed, with a median (IQR) of 11 (6-16) per patient over 6.6 (4.8-11) days. Concurrent NRS withdrawal intensity ratings were performed for 816 (78.3%) of the assessments. Table 3 shows the overall and age group-specific prevalence rates for each symptom. There was only a marginal age effect for the startle-to-touch symptom (chi-square=5.1, p=0.085).

Table 3. Withdrawal symptoms by age group.

(adjusted for clustering of observations within site and patient; n=1040)

	Overall	0-2 years	2.1 - 6 years	>6 years
From patient record, previous 12 hours % (CI)
Any loose /watery stools	20.1 (15.6,25.6)	18.4 (12.0,27.3)	19.9 (12.1, 30.9)	22.0 (14.5, 31.9)
Any vomiting/wretching/gagging	7.6 (4.5,12.6)	12.3 (6.5,22.5)	5.9 (2.5,13.4)	3.2 (1.3,7.6)
Temperature > 37.8°C	25.1 (19.3,31.9)	19.5 (12.0-30.2)	18.3 (11.2,28.4)	34.2 (23.8,46.3)
Respiratory rate often > baseline for this child	44.2 (37.0,51.8)	42.8 (31.7,54.7)	39.3 (25.6,54.8)	48.1 (35.7,60.7)
Required suctioning > 1 every 2 hrs	16.7 (11.5,23.6)	18.2 (9.7,31.3)	14.5 (7.1,27.4)	16.1 (9.1,26.9)
2 minute pre-stimulus observation % (CI)
State: SBS1≥ +1 or awake distressed	15.8 (11.9,20.6)	17.8 (12.2,25.2)	12.9 (7.5,21.2)	14.9 (8.6,24.5)
Tremor - moderate/severe	7.9 (5.1,12.0)	8.8 (4.6,16.0)	3.2 (1.0,9.7)	9.1 (4.5,17.3)
Any Sweating	13.2 (8.8,19.3)	13.3 (7.4,22.5)	7.0 (2.2,20.1)	15.9 (7.8,26.0)
Uncoordinated/repetitive movements - moderate/ severe	5.8 (3.8,8.8)	9.0 (5.7,14.0)	0.5 (0.1,3.9)	4.7 (2.0,10.5)
Yawning or sneezing - 2 or more times	9.1 (6.5,12.5)	9.9 (5.8,16.5)	13.4 (6.0,27.3)	6.1 (3.4,10.9)
1 minute stimulus observation % (CI)
Startle to touch - moderate/severe2	8.2 (5.9,11.3)	10.6 (6.8,16.1)	10.2 (5.0,19.7)	4.7 (2.3,9.0)
Pupils ≥ 4mm	19.3 (13.6,26.6)	13.3 (7.3,22.8)	19.5 (7.9,40.4)	25.7 (15.4,39.8)
Muscle tone - increased	16.6 (12.1,22.2)	21.1 (13.5,31.4)	9.7 (3.7,23.2)	14.7 (8.3,24.7)
State - SBS1 ≥+1 or awake and distressed	17.6 (13.6,22.5)	20.9 (15.5,27.5)	15.6 (8.7,26.4)	15.0 (8.4,25.3)
Tremor - moderate/severe	10.3 (7.2,14.5)	12.6 (7.2,21.0)	7.0 (2.5,18.3)	9.3 (5.2,16.0)
Any sweating	13.3 (8.7,19.4)	13.7 (7.8,22.9)	7.5 (2.6,20.0)	15.4 (7.4,29.4)
Uncoordinated/repetitive movements - Moderate/ Severe	7.5 (5.1,10.7)	10.8 (7.2,15.9)	4.0 (0.8,19.6)	5.4 (2.4,11.8)
Yawning or sneezing - 2 or more times	9.1 (6.5,12.4)	9.9 (5.8,16.5)	11.8 (5.1,25.1)	6.9 (3.8,12.1)
Post-stimulus recovery % (CI)
Time to gain calm state (SBS1 < 0) -    2-5 minutes    > 5 minutes	14.6 (10.6,19.9) 5.7 (3.7,9.2)	20.2 (13.3,29.5) 6.7 (3.7,12.1)	10.2 (5.2,19.2) 3.8 (1.0,13.2)	10.6 (6.1,17.8) 5.9 (2.5,13.4)
Total Score
WAT-1 = 0 (%)3	43.9	48.1	48.9	37.2
1	17.2	10.6	17.7	24.2
2	13.6	11.2	14.0	15.9
3	8.6	7.9	8.6	9.3
4	6.1	7.2	4.8	5.4
5	3.6	3.6	3.2	3.7
6	3.6	5.4	1.6	2.4
7+	3.4	6.0	1.2	1.9
Total WAT-1 (Median, IQR)	1.0 (0.0-3.0)	1.0 (0.0-3.0)	1.0 (0.0-2.0)	1.0 (0.0-2.00
WAT-1 Peak (Median, IQR)	3.0 (2.0-6.0)	4.5 (2.0-7.0)	3.0 (1.5-5.0)	4.0 (2.0-6.0)

WAT-1: Withdrawal Assessment Tool - 1; CI: Confidence interval; IQR: Interquartile range

Bold = in final WAT-1; Non-bold/light shaded items = eliminated from WAT-1

¹Curley et al. State behavioral scale: A sedation assessment instrument for infants and young children supported on mechanical ventilation. Pediatr Crit Care Med 2006;7(2):107-114.

²Startle to touch: greater than 6 years < other age groups, p=.09

³Total WAT-1 Score (%): greater than 6 years < other age groups, p=.0

The inter-item analyses revealed redundancy between the pre-stimulus and stimulus ratings for sweating, uncoordinated/repetitive movement, tremor, yawning, and behavioral state, with kappas ranging from .65 to .91 and crude percent agreement ranging from 92.2 to 98.1%. Redundant items were dropped from either pre-stimulus or stimulus observations to reduce patient disturbance and increase ease of symptom assessment (Table 3). Symptoms that occurred with relatively high prevalence among those with NRS of 0, or that least differentiated patients that received higher NRS withdrawal intensity scores (top 20^th percentile) from those with lower scores, included elevated respiratory rate, suctioning, and dilated pupils. Therefore, these three items were dropped. The resulting measure consisted of 11 items (12-point) scale, which we henceforth refer to as the WAT-1 (Table 4.). The correlation between the 19 original symptoms and the more parsimonious WAT-1 was .947.

Table 4. Withdrawal Assessment Tool Version 1 (WAT - 1) and Instructions.

© 2007 L.S. Franck and M.A.Q. Curley. All Rights reserved. Reproduced by permission of Authors

Patient Identifier
	Date:
	Time:
Information from patient record, previous 12 hours
Any loose /watery stools	No = 0 Yes = 1
Any vomiting/wretching/gagging	No = 0 Yes = 1
Temperature > 37.8°C	No = 0 Yes = 1
2 minute pre-stimulus observation
State	SBS1≤ 0 or asleep/awake/calm = 0 SBS1≥ +1 or awake/distressed = 1
Tremor	None/mild = 0 Moderate/severe = 1
Any sweating	No = 0 Yes = 1
Uncoordinated/repetitive movement	None/mild = 0 Moderate/severe = 1
Yawning or sneezing	None or 1 = 0 ≥2 = 1
1 minute stimulus observation
Startle to touch	None/mild = 0 Moderate/severe = 1
Muscle tone	Normal = 0 Increased = 1
Post-stimulus recovery
Time to gain calm state (SBS1≤ 0)	< 2min = 0 2 - 5min = 1 > 5 min = 2
Total Score (0-12)

Withdrawal Assessment Tool(WAT - 1) Instructions

• Start WAT-1 scoring from the first day of weaning in patients who have received opioids +/or benzodiazepines by infusion or regular dosing for prolonged periods (e.g., > 5 days). Continue twice daily scoring until 72 hours after the last dose.

• The Withdrawal Assessment Tool (WAT-1) should be completed along with the SBS¹ at least once per 12 hour shift (e.g., at 08:00 and 20:00 ± 2 hours). The progressive stimulus used in the SBS¹ assessment provides a standard stimulus for observing signs of withdrawal.

Obtain information from patient record (this can be done before or after the stimulus):

 ✓ Loose/watery stools: Score 1 if any loose or watery stools were documented in the past 12 hours; score 0 if none were noted.

 ✓ Vomiting/wretching/gagging : Score 1 if any vomiting or spontaneous wretching or gagging were documented in the past 12 hours; score 0 if none were noted

 ✓ Temperature > 37.8°C: Score 1 if the modal (most frequently occurring) temperature documented was greater than 37.8 °C in the past 12 hours; score 0 if this was not the case.

2 minute pre-stimulus observation:

 ✓ State: Score 1 if awake and distress (SBS¹: ≥ +1) observed during the 2 minutes prior to the stimulus; score 0 if asleep or awake and calm/cooperative (SBS¹≤ 0).

 ✓ Tremor: Score 1 if moderate to severe tremor observed during the 2 minutes prior to the stimulus; score 0 if no tremor (or only minor, intermittent tremor).

 ✓ Sweating: Score 1 if any sweating during the 2 minutes prior to the stimulus; score 0 if no sweating noted.

 ✓ Uncoordinated/repetitive movements: Score 1 if moderate to severe uncoordinated or repetitive movements such as head turning, leg or arm flailing or torso arching observed during the 2 minutes prior to the stimulus; score 0 if no (or only mild) uncoordinated or repetitive movements.

 ✓ Yawning or sneezing > 1: Score 1 if more than 1 yawn or sneeze observed during the 2 minutes prior to the stimulus; score 0 if 0 to 1 yawn or sneeze.

1 minute stimulus observation:

 ✓ Startle to touch: Score 1 if moderate to severe startle occurs when touched during the stimulus; score 0 if none (or mild).

 ✓ Muscle tone: Score 1 if tone increased during the stimulus; score 0 if normal.

Post-stimulus recovery:

 ✓ Time to gain calm state (SBS¹≤ 0): Score 2 if it takes greater than 5 minutes following stimulus; score 1 if achieved within 2 to 5 minutes; score 0 if achieved in less than 2 minutes.

Sum the 11 numbers in the column for the total WAT-1 score (0-12).

¹Curley et al. State behavioral scale: A sedation assessment instrument for infants and young children supported on mechanical ventilation. Pediatr Crit Care Med 2006;7(2):107-114.

Factor structure

A 4-factor solution provided the best overall conceptual fit, explaining 58% of the variance in analysis of all WAT-1 assessments. Motor-related symptoms (tremor, uncoordinated/repetitive movements, muscle tone, and startle) comprised the factor that accounted for the most variance. The second factor was comprised of behavioral state (pre-stimulus state and return to calm state), the third factor was autonomic related (temperature and sweating), and the fourth factor was comprised of gastrointestinal symptoms (stooling, vomiting) and yawning. The components of the factor solutions varied slightly by age group and explained a total of 61% (0-2 years), 65% (2.1-6 years) or 56% (>6 years) of the variance. The item factor loadings were slightly different for children over 6 years of age compared to the younger age groups. For example, in the older age group, motor and state related symptoms loaded on the same factor and yawning and startle did not meet the threshold for inclusion (factor loadings ≥.40) in any factor.

Construct validity

There was a high degree of convergence between the WAT-1 total scores and withdrawal intensity ratings (Spearman’s rho coefficient = .80). Examination of receiver-operator characteristic curves revealed that a WAT-1 score of 3 or higher had the best sensitivity and specificity in relation to an intensity rating indicating clinically significant withdrawal, i.e., 4 or higher (.872 and .880; Figure 1).

Figure 1. Receiver Operator Characteristic Curve: WAT-1 and NRS of withdrawal intensity¹ (n = 816).

Legend: Area under the curve: .944 ± .008 (SE), CI: .928 - .961, p < .001; WAT-1 score of 3 or higher had the best sensitivity and specificity in relation to NRS withdrawal intensity ratings of 4 or higher (.872 and .880, respectively)

We found that the 53 (64%) patients with higher peak WAT-1 scores (≥3) had greater cumulative opioid doses and longer duration of opioid treatment prior to tapering (Figure 2) compared to the 30 (36%) patients whose symptoms were less severe (WAT-1 < 3; Figure 2). Peak WAT-1 scores for each patient correlated moderately with the length of opioid therapy (r=.23, p=.04) and benzodiazepine therapy (r=.23, p=.04) prior to weaning and with the length of opioid weaning (r=.33, p=.003). Opioid weaning was completed in more patients with peak WAT-1 scores < 3 compared to those with higher peak scores (chi-square=4.3, p=.04). In addition, patients with higher peak WAT-1 scores (≥3) had a longer opioid weaning period (median (IQR)=13.0 (9.0-18.0) vs 8.0 (5.0-12.0) days; p=.004) as well as longer lengths of mechanical ventilation (median (IQR)=11.7 (8.2-15.6) vs 6.9 (5.4-9.6) days; p<.001), PICU stay (median (IQR)=17.0 (12.0-27.0) vs 10.5 (9.0-15.0) days; p<.001) and hospital stay (median (IQR)=29.0 (19.0-42.0) vs 20.0 (14.0-28.0) days; p=.01) than those with scores of <3. There were no differences in WAT-1 scores or intensity rating related to receiving other non-opioid or sedative drugs during the weaning period.

Figure 2. Comparison of opioid exposure levels by WAT-1 scores. (n=83).

Legend: Panel a. Greater median cumulative opioid dose for WAT-1 3+ vs WAT-1 < 3 (p<.002); Panel b. Longer median length of opioid treatment prior to tapering for WAT-1 3+ vs WAT-1 < 3 (p<.04).

Discussion

Within the context of a clinical trial where clinical parameters relating to analgesic and sedative administration and patient response were prospectively recorded over the duration of therapy, we were able to examine the psychometric properties of an instrument to quantify the prevalence and severity of withdrawal symptoms in children at high risk for development of iatrogenic withdrawal. From these data, we were able to construct a parsimonious instrument, the WAT-1 and to demonstrate its preliminary concurrent and predictive validity. This is the largest prospective investigation and most comprehensive analysis of opioid and benzodiazepine withdrawal symptoms in critically ill children.

The factor structure of the WAT-1 conformed to the main expected symptom clusters of motor disturbance, behavioral state disturbance, autonomic disturbance and gastrointestinal symptoms.^1;4;5 There were some differences in the frequency and pattern of symptoms based on age, which is to be expected given the differences in physical maturation across the age span of children in this study. Further research is needed to determine whether these differences can be accommodated in a single withdrawal assessment tool or if different variants of the tool for children of different age groups would provide better diagnostic accuracy.

The WAT-1 showed excellent sensitivity and specificity compared to NRS overall rating of withdrawal severity and greater validity than intensity ratings as demonstrated by its better performance in relation to known risk factors for iatrogenic withdrawal such as opioid exposure and length of therapy. We were unable to investigate differences between opioid and benzodiazepine withdrawal symptoms, since all patients received both drugs during the study. However, the validity analysis suggests that the WAT-1 is better at detecting symptoms of opioid rather than benzodiazepine withdrawal. This is not surprising given that benzodiazepine withdrawal is often described as more subtle and with fewer symptoms.¹⁸

We were also unable to examine relationships between the speed of opioid or benzodiazepine dose tapering and the emergence of withdrawal symptoms because of the variability of the weaning pattern over the observation periods. These questions are best investigated in a clinical trial with withdrawal as the primary endpoint, testing two or more distinct protocols for achieving analgesia and sedation weaning. Although previous literature has suggested weaning algorithms for children varying from 10-50% of the peak dose at 4-24 hourly intervals and the addition or substitution of other drugs, ^1;4;5;9 only one prospective trial used the NAS score and found no difference when patients were converted from fentanyl to methadone and weaned over a 5-day or 10-day period. ¹⁹ However, the NAS and its variants lacks sensitivity for the PICU setting, ¹¹ is burdensome (requiring 2 to four hourly assessments) and is subject to serial bias.

This study has some limitations, the most significant of which is the incomplete independence of the withdrawal symptom scoring and withdrawal intensity ratings. Although clinical judgment is often termed a ‘tin standard’, no validated biomarkers of iatrogenic withdrawal presently exist. The use of polypharmacy may also have confounded the analyses. However, clinician judgment about a patient’s immediate medical needs rightly over-ride the need for uniformity in practice required for instrument validation. Next, withdrawal symptom intensity may have been related to the patient’s primary medical condition and we did not search for these potential confounders. Lastly, the sample does not represent the entire population of children experiencing withdrawal symptoms, and replications of this study are needed in different samples and settings. However, in this initial study evaluating the psychometric properties of an instrument, having a probability sample (sampling of people) is less important than having the full range of symptom intensity (sampling of content) represented, as occurred in this study.²⁰

The main aims of a withdrawal assessment tool are to improve the detection and treatment of withdrawal symptoms before they compromise the patient’s clinical condition. Since the NAS was first shown to reduce the treatment time for neonates with prenatal drug exposure,¹⁰ the superiority of an assessment tool over subjective clinical assessment has been assumed but never established for other patient groups or settings. The lack of adequate measures of iatrogenic withdrawal and the need for such tools to guide clinical practice has been repeatedly highlighted.^5;9;21 The WAT-1 is a significant improvement over previous withdrawal symptom assessment scales in that it has fewer items that can be more objectively and feasibly measured. It is performed only twice a day compared to the usual 6 to 12 times per day for other symptom assessment scales, which increases the likelihood that it will be used in clinical practice. The assessment of the WAT-1 parameters is easily integrated into the standard start-of-shift nursing assessment. Training can be accomplished through brief written instruction and bedside demonstration and, consistent with most clinical activities, we recommend periodic training updates and inter-rater reliability checks. Further research is needed to confirm the cut-off values for the diagnosis of withdrawal and decisions about treatment. Trends and direction of movement in WAT-1 scores may be more important because of the highly individualized effects of withdrawal symptoms on a patient’s clinical recovery.

In summary, iatrogenic withdrawal is a common side effect of prolonged sedation in critically-ill pediatric patients that requires better methods of assessment and monitoring. The WAT-1 shows excellent preliminary psychometric performance when used to assess clinically important withdrawal symptoms in the PICU setting. It is also much simpler and efficient that previous assessment methods. Further psychometric evaluation is needed in other at-risk groups, such as neonatal intensive care unit patients.