Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Aug 1.
Published in final edited form as: Anesthesiology. 2020 Aug;133(2):293–303. doi: 10.1097/ALN.0000000000003367

“Three-phase” goal-directed fluid therapy for open radical cystectomy: a prospective randomized controlled trial

Vittoria Arslan-Carlon 1, Kay See Tan 1, Guido Dalbagni 1, Alessia C Pedoto 1, Harry W Herr 1, Bernard H Bochner 1, Eugene K Cha 1, Timothy F Donahue 1, Mary Fischer 1, S Machele Donat 1
PMCID: PMC8320377  NIHMSID: NIHMS1720444  PMID: 32472804

Abstract

Background:

Postoperative ileus is a common complication of intra-abdominal surgeries, including radical cystectomy with reported rates as high as 32%. Targeted individualized perioperative fluid management has been shown to improve postoperative ileus. However, it is difficult to generalize from results of prior studies since many of the earlier studies lacked standardized definitions of outcomes.

Methods:

This is a parallel-arm, double-blinded, single-center randomized trial to determine if goal-directed fluid therapy is superior to standard fluid therapy for patients undergoing open radical cystectomy. Primary outcome was postoperative ileus and secondary outcome was complications within 30 days post-surgery. Participants were ≥21 years old, had body mass index ≤45, and no active atrial fibrillation. The algorithm combines preoperative and postoperative stroke volume (SV) optimization and intraoperative SV variation minimalization to guide fluid administration. All patients were treated on a standardized postoperative enhanced recovery pathway.

Results:

Between August 2014 and April 2018, 283 radical cystectomy patients (142 goal-directed fluid therapy and 141 standard fluid therapy) were included in the analysis. Postoperative ileus occurred in 25% (36/142) of patients in the goal-directed fluid therapy arm and 21% (30/141) of patients in the standard arm (difference in proportions 4.1%; 95% CI −5.8 to 13.9; P=0.4). Other complications of interest occurred in similar proportions between the two arms (e.g., high-grade complications: 20/142 (14%) vs 23/141 (16%); difference in proportions, −2.2%; 95% CI −10.6 to 6.1; P=0.6), with the exception of acute kidney injury that was more frequent in the goal-directed fluid therapy arm (56% [80/142] versus 40% [56/141] in the standard arm; difference in proportions 16.6%; 95% CI 5.1 to 28.1; P=0.005; P=0.17 after adjustment for multiple-testing).

Conclusions:

In patients undergoing open radical cystectomy in a standardized postoperative enhanced recovery pathway, the addition of a three-phase goal-directed fluid therapy algorithm did not reduce the incidence of postoperative ileus.

INTRODUCTION

Postoperative ileus is a common complication affecting patients after intra-abdominal surgery,1,2 including radical cystectomy where the rate has been reported in the range of 2–32%.3 Due to the potential for postoperative ileus to prolong hospitalization, multiple studies have evaluated potential risk factors and perioperative and intraoperative care pathways to find preventative interventions. Most studies evaluating postoperative ileus in radical cystectomy have been retrospective.4,5

Intuitively, individualized goal-directed fluid therapy guided by patient hemodynamic values seems to be the logical approach to avoid the extremes of fluid administration that can be associated with postoperative complications. However, data on the effect of goal-directed fluid therapy are still inconclusive and even though some trials have shown benefits in outcome,69 other trials have shown no benefit or even a negative impact on renal function.10,11

More recently, the first large-scale randomized goal-directed fluid therapy study by Pearse et al10 included 734 patients undergoing major gastrointestinal surgery and did not show a statistically significant benefit for goal-directed fluid therapy; however, an updated meta-analysis including this population demonstrated benefit for goal-directed fluid therapy in preventing infectious complications in patients above the age of 65. The recently-published, prospective, randomized FEDORA trial9 included 500 patients undergoing major elective abdominal surgery and showed that Doppler-guided goal-directed hemodynamic therapy reduced postoperative complications and hospital length of stay but did not show a difference in mortality.

The Pillai12 and Wuethrich13 studies suggest that individualized goal-directed fluid therapy intraoperative fluid management in radical cystectomy patients may further reduce gastrointestinal and cardiac complications. Since our institution had already standardized postoperative care, but intraoperative care was largely practitioner-dependent, we designed a three-phase goal-directed fluid therapy algorithm combining stroke volume (SV) optimization and SV variation minimalization to guide fluid administration during perioperative care for patients undergoing open radical cystectomy. This prospective randomized trial aims to evaluate whether using goal-directed fluid therapy versus our institution’s standard fluid therapy during open radical cystectomy has an impact on postoperative ileus and perioperative outcomes.

METHODS

Study design and participants

This double-blinded, prospective randomized trial was conducted under a Memorial Sloan Kettering Cancer Center Institutional Review Board-approved protocol (ClinicalTrials.gov NCT02145871; principal investigator V. Arslan-Carlon; registration date May 23, 2014; protocol can be accessed by request). All eligible patients with planned open radical cystectomy at Memorial Sloan Kettering Cancer Center were approached. Exclusion criteria included age < 21 years, active atrial fibrillation, and body mass index above 45 due to the limitations of SV variation reading. Due to possible differences in complication rates between open and minimally invasive surgical approaches, the protocol was limited to patients undergoing open radical cystectomy.

Randomization and masking

A trained research assistant evaluated eligibility, written informed consent was obtained by a consenting professional, and patients were randomly assigned in a 1:1 ratio to the goal-directed fluid therapy or standard fluid therapy arm. Randomization was performed via the Memorial Sloan Kettering Clinical Research Database using randomly-sized permuted blocks; allocation was concealed by the database system. Patients and assessors of the primary outcome were blinded as to the study arm. Unblinding occurred after the trial was closed and data on all patients were collected.

Procedures — preoperative and intraoperative care

Patients in both arms received a radial artery catheter and were connected to an EV1000 clinical platform via a Flotrac sensor (Edwards Lifesciences, Irvine, CA).Treatment in the standard fluid therapy arm was based on Memorial Sloan Kettering historic fluid administration data for open radical cystectomy: maintenance of 10 ml ● kg−1 ● h−1 of balanced crystalloid solution (Normosol-R, ICU Medical Inc., Lake Forest, IL) with blood loss replaced 1:1 with albumin 5% or packed red blood cells to maintain a Hb ≥7 mg/dl.

All patients in the goal-directed fluid therapy arm underwent a passive leg raise in the operating room prior to induction to determine fluid responsiveness, based on SV augmentation of >10%. Patients with positive results from the passive leg raise were optimized with 250-ml balanced crystalloid boluses until their SV was no longer responsive. After induction, fluids were administered at 3 ml ● kg−1 ● h−1 and albumin 5% was administered to maintain SV variation below 13%, packed red blood cells were used instead of albumin to maintain Hb ≥7 mg/dl. No blood loss was replaced unless accompanied by an increase in SV variation.

All bowel anastomoses were stapled and performed in a standard side-to-side fashion using either a 60 mm or 80 mm gastrointestinal anastomosis stapler and a thoracoabdominal stapler. None of the anastomoses were hand sewn.

Two hundred twenty-five patients received an epidural; epidural infusion was started once specimen was removed and blood loss was finished. Infusions were standardized by the pain service and started at 6 ml/hr of bupivacaine 0.05% with 8 mcg/ml of hydromorphone; additional boluses of 6 ml every 30 minutes were permitted at the discretion of the anesthesia practitioner.

The anesthesia team was blinded to the Flotrac reading in the standard fluid therapy arm. To keep uniformity of procedures, the anesthesiologists participating in the trial were limited to four; if none were available, the patient was excluded from the study population as pre-specified in the protocol.

Procedures — postoperative care

All patients were transported to the postanesthesia care unit (PACU) unless the intensive care unit was indicated due to intraoperative events or preoperative comorbidities. The protocol fluid administration continued for the first 6 h in PACU, with all patients in the standard fluid therapy arm receiving 1.5 ml ● kg−1 ● h−1 of balanced crystalloid solution, and those in the goal-directed fluid therapy arm receiving 1 ml ● kg−1 ● h−1 of maintenance, and any additional boluses given based on SV optimization. In cases of high potassium levels, balanced crystalloid solution was substituted with normal saline. All patients received colloid 250-ml boluses for systolic blood pressure <90 mmHg and/or urine output <0.5 ml ● kg−1 ● h−1 over 2 h. Patients were discharged from the PACU at the end of the 6-h protocol (or longer if required by preoperative comorbidities).

On the floor, all patients were treated on our standardized postoperative enhanced recovery pathway (outline shown in Supplemental Figure S1), adjusted only as necessary for individual allergies, renal function, medical comorbidities, and acute medical events.

Outcome measures

The primary endpoint of postoperative ileus was defined as intolerance of oral intake by postoperative day 5, the cessation of diet, or placement of a nasogastric tube for clinical signs or symptoms associated with postoperative ileus, including one or more of the following: nausea, emesis, abdominal bloating or distension, or excessive burping.5 In addition, to separate causative versus secondary postoperative ileus we added: “primary postoperative ileus” as a gastrointestinal dysfunction that occurred in the absence of major grade 3–5 surgical or medical complications, defined based on the modified Clavien system,5,14,15 Although this was not a protocol-specified endpoint, we defined and considered “primary postoperative ileus” as a secondary outcome before unblinding and data analysis. Protocol-defined secondary outcomes were: total hospitalization fluid, blood transfusion rates, total dose of vasoactive agents, and pattern of overall and specific 30-day postoperative complications. Additional secondary outcomes were vasopressor use in the operating room, high-grade complications, and length of stay. Thirty-day complications were captured prospectively using the modified Clavien system14 and the definitions described in our group’s prior paper.5

Renal function was assessed by both serum creatinine and calculated estimated glomerular filtration rate using both the Modification of Diet in Renal Disease (MDRD) and Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) formulas; values were recorded at baseline (within one month before surgery) and at the time of hospital discharge, in addition to frequent serum creatinine evaluations during the hospitalization. We used the standard National Kidney Center classification system to describe the stages of chronic kidney disease.16 For the purposes of the study, pre-existing renal insufficiency was defined as baseline renal function <60 ml/min, consistent with stage 2 chronic kidney disease; the 60-ml/min cutoff is often used to determine if patients are candidates for neoadjuvant chemotherapy.

Statistical analysis

All analyses were performed in a modified intention-to-treat population, which included all patients who had undergone both randomization and anesthesia with EV1000 monitoring for eligible surgery. All evaluable patients were followed for 30 days postoperatively and none were lost to follow-up.

With a historical institutional postoperative ileus rate of 32%17 and a two-sided type I error of 0.05, we calculated that 283 evaluable patients would provide 80% power to detect a 15% absolute difference in the proportions of patients with postoperative ileus between the two arms (hypothesized postoperative ileus rate of 17% in the goal-directed fluid therapy arm). This sample size also allowed for an interim analysis, using O’Brien-Fleming boundaries for both efficacy and futility. Pooled variance and the Casagrande-Pike-Smith continuity correction were utilized in the sample size calculation. The interim analysis was performed once the accrual reached 144 evaluable patients, but since it did not meet protocol-specified futility or efficacy thresholds, the trial continued to full enrollment.

The distributions of patient characteristics and outcomes were summarized as number (proportion) for categorical factors and median (interquartile range) for continuous factors. The primary outcome was proportion of patients with postoperative ileus within 30 days of operation, which was compared between the randomized arms using the chi-square test and quantified as difference in proportions with the corresponding 95% CI. All binary secondary outcomes were assessed similarly. Secondary outcomes measured on a continuous scale were compared between the randomized arms using the Wilcoxon rank-sum test and quantified as differences in means with corresponding 95% CI. A natural-log transformation was applied to outcomes that displayed a skewed distribution. In these instances, the estimated size of differences on the log-scale were converted to the ratio of means on the original scale. The widths of the confidence intervals have not been adjusted for multiple-testing, so the intervals should not be used for inference. No stratification was used in the analyses. Neither multivariable analyses to adjust for preoperative risk nor pre-planned sub-group analyses were planned or performed.

Multiple testing was addressed by applying the Holm-Bonferroni adjustment to p-values from all secondary outcomes with a family-wise significance level of 0.049 to account for the interim analysis. Statistical tests are two-sided. Analyses were conducted with Stata 13.1 (StataCorp, College Station, TX, USA).

RESULTS

Patients

Between August 5, 2014 and April 9, 2018, 320 patients soon to undergo open radical cystectomy were consented to the protocol and randomized (Figure 1). Twenty-nine were excluded after randomization based on protocol-defined exclusion criteria: 21 due to unavailability of participating anesthesiologists, 8 due to change in surgical plan, 4 patients withdrew consent, and 4 were found ineligible after consent due to atrial fibrillation on preoperative electrocardiogram. In total, 283 patients were considered evaluable and included in the analyses (142 in the goal-directed fluid therapy arm and 141 in the standard fluid therapy arm).

Figure 1.

Figure 1.

CONSORT diagram of patient flow.

Baseline demographics, clinical characteristics, and preoperative risk factors for postoperative ileus were similar for both arms (see Table 1). Intraoperative characteristics are shown in Table 2.

Table 1:

Demographic and preoperative characteristics of patients

Combined N=283 Goal-directed fluid therapy N=142 Standard fluid therapy N=141
Sex
 Female 62 (22%) 28 (20%) 32 (23%)
 Male 221 (78%) 112 (80%) 109 (77%)
Age at surgery (years) 69 (63–76) 70 (63–77) 69 (62–75)
Charlson Comorbidity Index (excluding 2 points for cancer) 1 (0–2) 1 (0–2) 1 (0–2)
Charlson Comorbidity Index adjusted for age 4 (3–6) 5 (3–6) 4 (3–6)
Anesthesia Category (ASA score 1–2* vs ASA score 3–4): highest value between presurgical testing and preoperative day
 ASA 1–2* 85 (30%) 44 (31%) 41 (29%)
 ASA 3–4 198 (70%) 98 (69%) 100 (71%)
Body mass index at surgery 28.7 (25.4–31.8) 28.7 (25.0–32.0) 28.7 (25.8–31.4)
Body mass index above 30 (obesity) 111 (39%) 57 (40%) 54 (38%)
Number of patients with renal insufficiency by eGFR<60 ml/min 95 (34%) 42 (30%) 53 (38%)
Number of patients with abnormal creatinine 48 (17%) 20 (14%) 28 (20%)
Preoperative albumin 5% below 3 g/dl 2 (0.7%) 2 (1.4%) 0 (0%)
Smoking within the 6 months before surgery 34 (12%) 14 (10%) 20 (14%)
 Number of pack-years (N=282) 15 (0–40) 15 (0–40) 15 (0–40)
History of COPD (emphysema, asthma, chronic bronchitis) 46 (16%) 29 (20%) 17 (12%)
History of coronary artery disease 58 (20%) 37 (26%) 21 (15%)
 Myocardial infarction in past 16 (5.7%) 12 (8.5%) 4 (2.8%)
Longstanding arrhythmia 9 (3.2%) 7 (4.9%) 2 (1.4%)
Prior venous embolic event (deep vein thrombosis, pulmonary embolism) 36 (13%) 16 (11%) 20 (14%)
Non-insulin-dependent diabetes 49 (17%) 24 (17%) 25 (18%)
Insulin-dependent diabetes 12 (4.2%) 6 (4.2%) 6 (4.3%)
Diabetes-related peripheral neuropathy 6 (2.1%) 3 (2.1%) 3 (2.1%)
Hyperlipidemia 179 (63%) 84 (59%) 95 (67%)
Hypertension 174 (61%) 89 (63%) 85 (60%)
History of colitis 20 (7.1%) 9 (6.3%) 11 (7.8%)
History of gastroesophageal reflux disease 94 (33%) 46 (32%) 48 (34%)
Prior pelvic surgery 102 (36%) 46 (32%) 56 (40%)
History of prior bowel or abdominal surgery 54 (19%) 29 (20%) 25 (18%)
Prior abdominal or pelvic radiation therapy 30 (11%) 14 (10%) 16 (11%)
Received neoadjuvant chemotherapy 124 (44%) 65 (46%) 59 (42%)

Values are presented as n (%) or median (interquartile range).

*

No patient had an American Society of Anesthesiologists (ASA) score of 1.

Prior pelvic surgery included radical retropubic prostatectomy and total abdominal hysterectomy.

Abbreviations: COPD, chronic obstructive pulmonary disease; eGFR, estimated glomerular filtration rate.

Table 2:

Intraoperative management

Combined N=283 Goal-directed fluid therapy N=142 Standard fluid therapy N=141 P
Type of diversion 0.3
 Ileal conduit 180 (64%) 85 (60%) 95 (67%)
 Neobladder 99 (35%) 54 (38%) 45 (32%)
 Continent stomal diversion 4 (1.4%) 3 (2.1%) 1 (0.7%)
Ureteral stents used 0.031
 None 76 (27%) 46 (32%) 30 (21%)
 One side 15 (5.3%) 4 (2.8%) 11 (7.8%)
 Both sides 192 (68%) 92 (65%) 100 (71%)
Size of bowel anastomotic stapler 0.084
 60 mm 144 (51%) 65 (46%) 79 (56%)
 80 mm 139 (49%) 77 (54%) 62 (44%)
Length of surgery in minutes 314.0 (275.0–355.0) 319.0 (288.0–352.0) 310.0 (267.0–358.0) 0.6
Length of anesthesia in minutes 402.0 (350.0–445.0) 416.0 (370.0–444.0) 391.0 (337.0–445.0) 0.1
Extubated in operating room 278 (98%) 140 (99%) 138 (98%) 0.6
Extubated in recovery room 5 (1.8%) 2 (1.4%) 3 (2.1%)
Type of anesthesia (general, epidural, combined) NA
 General 283 (100%) 142 (100%) 141 (100%)
Epidural for postop pain control (N=282) 0.7
  No 57 (20%) 30 (21%) 27 (19%)
  Yes 225 (80%) 112 (79%) 113 (81%)

Values are presented as n (%) or median (IQR).

Abbreviations: N/A, not available.

Fluid management (volume and type), weight differences and changes, and fluid balance characteristics are shown in Table 3. The goal-directed fluid therapy arm reported lower total fluid output (median [interquartile range]) than the standard fluid therapy arm (13380 mL [10405–19093] vs 15445 mL [11920–21015]; P=0.044) and higher colloid intake intraoperatively (1000 mL [750–1250] vs 750 mL [500–1000]; P=0.005) though not over the whole protocol period, which included the 6 h of recovery (1000 mL [750–1500] vs 975 mL [500–1350]; P=0.053). The goal-directed fluid therapy arm also had a lower intraoperative and 6-h recovery room crystalloid intake (median [interquartile range] 2892 mL [2340–3450] vs 5580 mL [4650–6730]; P<0.0001) compared to the standard fluid therapy arm. Total dose of intraoperative vasopressors (ephedrine or phenylephrine) was comparable between arms (Tables 3 and 4), similarly for proportions of patients who received vasopressors in the operating room or in the PACU (Table 4). All but 5 patients were extubated in the operating room (Table 2). Transfusion rates were also comparable between arms (Table 3).

Table 3:

Administered fluid volume, postoperative weight change, and blood loss

Combined N=283 Goal-directed fluid therapy N=142 Standard fluid therapy N=141 P
FLUID VOLUMES
Protocol crystalloid (ml) 4019 (2890–5730) 2892 (2340–3450) 5580 (4650–6730) <0.0001
Protocol colloid (albumin 5%, PRBC, FFP and PLT) (ml) 1000 (750–1500) 1000 (750–1500) 975 (500–1350) 0.053
OR crystalloid (ml) 2900 (1750–4900) 1800 (1400–2220) 4850 (3800–5900) <0.0001
OR colloid (albumin 5%, PRBC, FFP and PLT) (ml) 800 (500–1250) 1000 (750–1250) 750 (500–1000) 0.005
PACU crystalloid (ml) 840 (700–1090) 1018 (791–1370) 750 (650–875) <0.0001
PACU colloid (albumin 5%, PRBC, FFP and PLT) (ml) 0 (0–250) 0 (0–250) 0 (0–250) 0.8
Total fluid intake during hospitalization (ml) 12768 (9658–17224) 12657 (9912–17280) 13072 (9363–17157) 0.7
Total fluid output during hospitalization (ml) 14045 (10875–20335) 13380 (10405–19093) 15445 (11920–21015) 0.044
Net fluid during hospitalization (ml) −1689 (−4027–693) −1296 (−3146–933) −1986 (−5337–605) 0.020
Number of patients with negative fluid balance 191 (67%) 93 (65%) 98 (70%) 0.5
WEIGHT
Peak weight (kg) during hospitalization 86.1 (76.0–98.3) 86.2 (74.0–98.6) 86.1 (77.0–96.9) 0.8
Maximum weight change (kg) 2.9 (1.4–4.7) 2.7 (1.0–4.5) 3.0 (1.7–4.8) 0.10
TRANSFUSION MANAGEMENT
Total EBL (ml) 700 (480–1000) 600 (400–900) 750 (500–1000) 0.020
Number of patients with EBL > one liter 57 (20%) 26 (18%) 31 (22%) 0.4
Number of patients transfused PRBC in OR 43 (15%) 18 (13%) 25 (18%) 0.2
Number of patients transfused PRBC in RR 70 (25%) 31 (22%) 39 (28%) 0.3
Number of patients transfused PRBC on the floor 90 (32%) 47 (33%) 43 (30%) 0.6
Number of patients transfused PRBC during hospitalization 142 (50%) 69 (49%) 73 (52%) 0.6
Number of patients transfused > 4 units PRBC during hospitalization 10 (3.5%) 3 (2.1%) 7 (5.0%) 0.2
Number of patients transfused FFP during hospitalization 12 (4.2%) 4 (2.8%) 8 (5.7%) 0.2
Number of patients transfused PLT during hospitalization 10 (3.5%) 4 (2.8%) 6 (4.3%) 0.5

Values are presented as n (%) or median (IQR). Colloids are defined as albumin 5% + transfusion volume. Fluids reflect the crystalloids + colloids.

Abbreviations: EBL, estimated blood loss; FFP, fresh frozen plasma; OR, operating room; PACU, postanesthesia care unit; PLT, platelets; PRBC, packed red blood cells; RR, recovery room.

Table 4:

Primary and secondary outcomes

Goal-directed fluid therapy N=142 Standard fluid therapy N=141 P* Effect size (95% CI)
PROTOCOL-DEFINED PRIMARY OUTCOME
POI – either primary or secondary 36 (25%) 30 (21%) 0.4 4.1% (−5.8 to 13.9)
SECONDARY OUTCOMES
Primary POI (absence of other ≥Grade 3 complications) 28 (20%) 24 (17%) 0.6 2.7% (−6.3 to 11.7)
Required NGT replacement during hospitalization or prolonged use postop 22 (15%) 23 (16%) 0.9 −0.8% (−9.3 to 7.7)
Total fluid intake during hospitalization (ml) 12657 (9912–17280) 13072 (9363–17157) 0.7 Ratio: 0.99 (0.89 to 1.12)
Number of patients transfused PRBC during hospitalization 69 (49%) 73 (52%) 0.6 −3.2% (−14.8 to 8.5)
Dose of vasopressors given intraoperatively
 Ephedrine dose (mg) 5 (0–20) 5 (0–20) 0.8 Ratio: 0.96 (0.67 to 1.37)
 Phenylephrine dose (mcg) 80 (0–200) 100 (0–240) 0.3 Ratio: 0.68 (0.35 to 1.33)
Number of patients who received vasopressors in OR
  Ephedrine 82 (58%) 84 (60%) 0.8 −1.8% (−13.3 to 9.6)
  Phenylephrine 79 (56%) 88 (62%) 0.2 −6.8% (−18.2 to 4.7)
Number of patients who received vasopressors in PACU 4 (2.8%) 7 (5.0%) 0.4 −2.1% (−6.6 to 2.4)
Any complications within 30 days of surgery 136 (96%) 125 (89%) 0.085 5.7% (−0.8 to 12.2)
  High grade (grade 3–5) complication 20 (14%) 23 (16%) 0.6 −2.2% (−10.6 to 6.1)
  Complications post discharge (at 30-day follow-up) 72 (51%) 69 (49%) 0.8 1.8% (−9.9 to 13.4)
Symptomatic postop arrhythmia 10 (7.0%) 10 (7.1%) >0.99 −0.05% (−6.0 to 5.9)
Postop CHF 1 (0.7%) 2 (1.4%) 0.6 −0.7% (−3.1 to 1.7)
Postop pulmonary issue (atelectasis, pneumonia, or hypoxia) 28 (20%) 33 (23%) 0.5 −3.7% (−13.2 to 5.9)
Postop pulmonary embolus 4 (2.8%) 9 (6.4%) 0.2 −3.6% (−8.4 to 1.3)
Postop DVT 4 (2.8%) 5 (3.5%) 0.7 −0.07% (−4.8 to 3.4)
Postop acute kidney injury 80 (56%) 56 (40%) 0.005§ 16.6% (5.1 to 28.1)
Discharge creatinine 0.9 (0.8–1.1) 0.9 (0.7–1.1) 0.4 Ratio: 1.04 (0.98 to 1.09)
Discharge eGFR (MDRD) 80 (64–95) 81 (62–98) 0.8 Ratio: 1.09 (0.96 to 1.24)
Urinary obstruction 4 (2.8%) 6 (4.3%) 0.5 −1.4% (−5.7 to 2.9)
Urinary leak 8 (5.6%) 6 (4.3%) 0.6 1.4% (−3.7 to 6.4)
Symptomatic postop UTI 25 (18%) 23 (16%) 0.8 1.3% (−7.4 to 10.3)
Intra-abdominal abscess 12 (8.5%) 11 (7.8%) 0.8 1.3 (−7.4 to 10.0)
Sepsis 14 (10%) 14 (10%) >0.99 −0.07% (−7.0 to 6.9)
Wound infection 32 (23%) 42 (30%) 0.2 −7.3 (−17.5 to 3.0)
Wound dehiscence 3 (2.1%) 5 (3.5%) 0.5 −1.4% (−5.3 to 2.4)
Length of stay in hospital (days) 7 (6–9) 7 (6–10) 0.6 Ratio: 0.99 (0.91 to 1.08)
Required UCC visit within 30 days of surgery 59 (42%) 50 (35%) 0.3 6.1% (−5.2 to 17.4)
Required readmit to hospital within 30 days 35 (25%) 35 (25%) >0.99 0.2% (−10.2 to 9.9)
Required return to OR as inpatient 6 (4.2%) 2 (1.4%) 0.2 2.8% (−1.0 to 6.6)
Required OR procedure within 30 days as outpatient or during MSK readmission 0 (0%) 3 (2.1%) 0.08 −2.1% (−4.5 to 0.3)
Required IR procedure during primary hospitalization 6 (4.2%) 7 (5.0%) 0.8 −0.7% (−5.6 to 4.1)
Required outpatient IR procedure within 30 days of surgery 10 (7.0%) 11 (7.8%) 0.8 −0.8% (−6.9 to 5.3)

Values are presented as n (%) or median (IQR).

*

P-values (before multiple-testing adjustments) were calculated using Chi-square test for categorical outcomes and Wilcoxon rank-sum test for continuous outcomes.

Effect sizes were based on difference in proportions for binary outcomes and ratio of means for continuous outcomes. The width of 95% confidence intervals have not been adjusted for multiple-testing.

After adjustment for multiple-testing, the P-value = 0.9 for any complications within 30 days of surgery.

§

After adjustment for multiple-testing, the P-value = 0.17 for acute kidney injury.

Abbreviations: CHF, congestive heart failure; CI, confidence interval; DVT, deep vein thrombosis; eGFR, estimated glomerular filtration rate; IR, interventional radiology; MDRD, Modification of Diet in Renal Disease formula; MSK, Memorial Sloan Kettering Cancer Center; NGT, nasogastric tube; OR, operating room; PACU, postanesthesia care unit; PRBC, packed red blood cells; POI, postoperative ileus; UCC, urgent care center; UTI, urinary tract infection.

Postoperative ileus

The overall postoperative ileus rate in this study was 23.3% (66/283) with 68% (45/66) of those suffering from postoperative ileus requiring nasogastric tube decompression for treatment (Table 4). The incidence of postoperative ileus was 25% (36/142) in the goal-directed fluid therapy arm and 21% (30/141) in the standard arm (difference in proportions 4.1%; 95% CI −5.8 to 13.9; P=0.4). The arms were also similar in proportion of patients requiring nasogastric tubes (15% [22/142] in goal-directed fluid therapy arm versus 16% [23/141] in standard arm; difference in proportions −0.8, 95% CI −9.3 to 7.7; P=0.9). Primary postoperative ileus (postoperative ileus in the absence of a major grade 3–5 complication) occurred in 18% (52/283) of patients overall, with no statistically significant difference between treatment arms: 20% (28/142) in the goal-directed fluid therapy arm versus 17% (24/141) in the standard arm (difference in proportions 2.7%; 95% CI −6.3 to 11.7; P=0.6).

Other complications

Overall, 92% (261/283) of patients experienced at least one complication (grade 1–5) within 30 days of surgery, with a higher rate in the goal-directed fluid therapy arm (96% [136/142] versus 89% [125/141] in the standard fluid therapy arm (difference in proportions 5.7%; 95% CI −0.8 to 12.2; P=0.085) (Table 4). However, only 16% (44/283) of all patients experienced a high-grade (grade 3–5) complication within 30 days of surgery: 20/142 (14%) in goal-directed fluid therapy arm vs 23/141 (16%) in standard arm (difference in proportions −2.2%; 95% CI −10.6 to 6.1; P=0.6). The proportions of patients with specific complications were similar between the two treatment arms (Table 4). In addition, 50% (141/283) of patients suffered a complication within 30 days after their hospital discharge, 39% (109/283) of patients required a visit to urgent care, and 25% (70/283) required readmission. The 30-day perioperative mortality rate was 0.7% (2/283).

Acute kidney injury was more frequent in the goal-directed fluid therapy arm (56% [80/142] versus 40% [56/141] in the standard fluid therapy arm; difference in proportions 16.6%; 95% CI 5.1 to 28.1; P=0.005; P=0.17 after adjustment for multiple-testing) (Table 4), but all patients recovered to their baseline renal function by the time of hospital discharge as reflected by serum creatinine and glomerular filtration rate; no patient required dialysis. The imbalance in acute kidney injury incidence also accounted for higher rate of overall genitourinary complications in the goal-directed fluid therapy arm (Table 5). Overall, 34% (95/283) of patients had preoperative renal insufficiency defined as estimated glomerular filtration rate <60 ml/min, and this occurrence was similar between the two arms: 42/142 (30%) in the goal-directed fluid therapy group vs 53/141 (38%) in the standard group (Table 1). Unlike some of the large abdominal surgery trials, we did not find statistically significant differences between treatment arms in terms of wound infection (23% [32/142] in goal-directed fluid therapy arm versus 30% [42/141] in standard arm; difference in proportions −7.3%; 95% CI −17.5 to 3.0; P=0.2) or intra-abdominal abscess (8.5% [12/142] in goal-directed fluid therapy arm versus 7.8% [11/141] in SFT arm; difference in proportions 1.3%; 95% CI −7.4 to 10.0; P=0.8). In exploratory analyses, the primary endpoint, postoperative ileus incidence, was also evaluated in relation to single preoperative comorbidities but no statistically significant association was found (Table 6).

Table 5.

Frequency of 30-day postoperative complications by category

Complication Category Overall N=283 Goal-directed fluid therapy N=142 Standard fluid therapy N=141 P
Surgical 11 (3.9%) 4 (2.8%) 7 (5.0%) 0.3
Wound 104 (37%) 46 (32%) 58 (41%) 0.13
Pulmonary 56 (20%) 27 (19%) 29 (21%) 0.7
Neurologic 64 (23%) 39 (27%) 25 (18%) 0.050
Genitourinary 144 (51%) 84 (59%) 60 (43%) 0.005
Infection 76 (27%) 39 (27%) 37 (26%) 0.8
Gastrointestinal 103 (36%) 55 (39%) 48 (34%) 0.4
Cardiac 61 (22%) 31 (22%) 30 (21%) 0.9
Bleeding 106 (37%) 54 (38%) 52 (37%) 0.8
Miscellaneous 29 (10%) 14 (10%) 15 (11%) 0.8
Thromboembolic 20 (7.1%) 7 (4.9%) 13 (9.2%) 0.2

Patients were recorded more than once if they had more than one complication within a category.

Please refer to Shabsigh et al5 for specific complications in each category.

Table 6.

Relationship of preoperative comorbidities to postoperative ileus

Postoperative ileus (N=66; 23%) No postoperative ileus (N=217; 77%) P
History of COPD (emphysema, asthma, chronic bronchitis) 12 (18%) 34 (16%) 0.6
History of coronary artery disease 9 (14%) 49 (23%) 0.11
  Myocardial infarction in past 3 (4.5%) 13 (6.0%) 0.7
Longstanding arrhythmia 3 (4.5%) 6 (2.8%) 0.5
Prior venous embolic event (DVT, PE) 8 (12%) 28 (13%) 0.9
Diabetes
  Non-insulin-dependent diabetes 10 (15%) 39 (18%) 0.6
 Insulin-dependent diabetes 3 (4.5%) 9 (4.1%) 0.9
Peripheral neuropathy 0.8
 None 54 (82%) 182 (84%)
 Diabetes-related neuropathy 1 (1.5%) 5 (2.3%)
 Non-diabetes-related neuropathy 11 (17%) 30 (14%)
Hyperlipidemia 43 (65%) 136 (63%) 0.7
Hypertension 39 (59%) 135 (62%) 0.6
History of colitis 3 (4.5%) 17 (7.8%) 0.4
History of gastroesophageal reflux disease 20 (30%) 74 (34%) 0.6
Prior pelvic surgery* 21 (32%) 81 (37%) 0.4
History of prior bowel or abdominal surgery 11 (17%) 43 (20%) 0.6
Prior abdominal or pelvic radiation therapy 10 (15%) 20 (9.2%) 0.2
Charlson Comorbidity Index 1 (0–2) 1 (0–2) 0.8

Values are presented as n (%) or median (IQR).

*

Prior pelvic surgery included radical retropubic prostatectomy and total abdominal hysterectomy.

Abbreviations: COPD, chronic obstructive pulmonary disease; DVT, deep vein thrombosis; PE, pulmonary embolism.

Length of stay was similar between the two treatment arms: median (interquartile range), 7 (6–9) days in goal-directed fluid therapy arm versus 7 (6–10) days in standard fluid therapy arm (ratio of means 0.99; 95% CI 0.91 to 1.08; P=0.6). The incidence of urgent care visits was not statistically significantly different between the two arms (42% [59/142] for goal-directed fluid therapy versus 35% [50/141] for standard fluid therapy; difference in proportions 6.1%; 95% CI −5.2 to 17.4; P=0.3).

DISCUSSION

This prospective randomized trial failed to demonstrate an advantage in using goal-directed fluid therapy to prevent postoperative ileus in patients undergoing radical cystectomy on an enhanced recovery pathway.

Postoperative ileus is one of the most common postoperative complications following radical cystectomy, with a reported incidence of 2–32% in a recent collaborative review of radical cystectomy series, including the Memorial Sloan Kettering experience.3 The incidence of POI and other complications can be affected by the patient population and their comorbidities, the time period examined, the definition utilized, method and quality of data collection, and the use of perioperative pathways.2,18 We designed this study based on our institution’s retrospective data for postoperative ileus following radical cystectomy that demonstrated rates ranging from 25% to 32% between 2000 and 2015. In this protocol, as in all recent radical cystectomy protocols at Memorial Sloan Kettering, we used standardized definitions of complications and a prospective data capture methodology as previously described.5,19

Review of the surgical literature reveals a variety of hypothesized risk factors for postoperative ileus associated with patient characteristics and perioperative care. In this study, individual risk factors for postoperative ileus were captured and analyzed, but no differences were observed between the goal-directed fluid therapy and standard fluid therapy arms, as shown in Table 1.

To date, our study is the largest single-population randomized goal-directed fluid therapy study evaluating perioperative outcomes in patients undergoing radical cystectomy for bladder cancer. Furthermore, the three-part algorithm designed to optimize fluid status has never been described before. Using SV before induction of anesthesia to establish normovolemia, using SV variation minimalization intraoperatively to maintain the normovolemia, and optimizing SV again once the patient reaches recovery, insures a more comprehensive approach than has been previously described to obtain a normovolemic status in the whole perioperative period.

In addition, our well-established postoperative enhanced recovery pathways and prospective complication reporting methodology, which have been in place since 2000 and 1995, respectively, and updated as evidentiary data are available, should minimize the effect on outcomes. In contrast to other radical cystectomy studies such as the Pillai et al12 study of 66 patients using Doppler-optimized intraoperative fluid management and the Wuethrich et al20 study of 166 patients using restrictive deferred hydration combined with pre-emptive norepinephrine infusion during radical cystectomy, we did not find that goal-directed fluid therapy was associated with any improvement in perioperative postoperative ileus, length of stay, or other perioperative/postoperative outcomes. When it comes to secondary outcomes, our findings are more in line with a fairly recent randomized trial in colorectal surgery that showed no difference in perioperative outcomes with goal-directed fluid therapy.15 Unlike the FEDORA trial,9 which is a more recent study comparing the use of goal-directed hemodynamic therapy to predefined standard care and which showed an outcome benefit for the use of goal-directed therapy, our results did not confirm a benefit and actually showed an increased risk of acute kidney injury with goal-directed therapy. This dissimilarity could be attributed to the different algorithm used in the FEDORA trial, with a more comprehensive hemodynamic approach to optimization using inotropic and pressure support driven by the algorithm.9 Another recent study (RELIEF), reported by Myles et al,11 compared the use of liberal vs restricted fluid management in major abdominal surgery and found no difference in disability-free survival at 1 year, but found an increased rate of acute kidney injury in the restrictive arm. Our study’s algorithm more closely resembled the algorithm in the RELIEF trial,11 with additional fluid in the restrictive arm driven by cardiac output monitoring, than the FEDORA9 algorithm. This could explain the similarity of our results to RELIEF and the dissimilarity with FEDORA. Moreover, the RELIEF liberal arm had a fluid administration very similar to our standard arm; in both our data and RELIEF there was no harm in the more liberal fluid administration: this might suggest that the initial teaching of fluid restriction in Enhanced Recovery After Surgery pathways might not be beneficial.11

Differences in postoperative ileus and perioperative outcomes between studies may also be partly explained by variations between studies in definitions, data capture, postoperative enhanced recovery pathways, time of follow-up, and patient comorbidities.18 For instance, the Wuethrich study13 never clearly defined what they considered postoperative ileus, and had a relatively high 22% rate of “constipation” (also not defined) in the control arm. The Pillai study12 defined ileus by the subjective measures “absence of bowel sound with a painful abdomen,” with no delineation of time to tolerance of oral intake, radiographic parameters, or intervention (ie, nasogastric tube use) parameters, which may have led to an underestimated ileus rate. We had a highly comorbid population (Table 1), with 70% of our population having an American Society of Anesthesiologists (ASA) score of 3–4, the overall population having an age-adjusted Charlson Comorbidity Index ≥4, and few medical restrictions to study entry (only active atrial fibrillation, or body mass index >45 due to the limitations of SV variation reading). Despite this, there were no statistically significant differences in high-grade complications (16% overall) or categories of complications between treatment groups. There was a higher incidence of transient acute kidney injury in the goal-directed fluid therapy group (56% vs 40%; multiple-testing adjusted P=0.17).

As in the recent goal-directed fluid therapy study by Gomez-Izquierdo,15 we also showed a statistically significant difference in intraoperative fluid administration, but when comparing the total fluid administration during the hospital stay, the difference disappears even more dramatically in our population, raising the question whether perioperative fluid administration is too small of a proportion compared to the full hospital stay to make a considerable outcome difference. This would strengthen the theory that the success in postoperative enhanced recovery pathways lies in the multiple changes and not a solitary intervention.

A more in-depth analysis of the overall population regardless of treatment arm showed no association between any of the preoperative comorbidities and the occurrence of postoperative ileus (Table 6). This would confirm that no prior condition or surgery predisposes any patient to developing postoperative ileus, and as such we cannot establish any early interventions for high-risk patients based on preoperative comorbidities.

There are several limitations in this study. This is a single-center trial with a very homogeneous population and a single surgery; however, this actually reduces variability in algorithm execution, allowing us to draw more precise conclusions regarding fluid administration and its effects. The algorithm used intraoperatively was based on SV variation minimalization. More recent studies have used SV optimization even during the intraoperative phase; however, our thought was that SV variation was easier to implement in the operating room and compliance to the execution of the algorithm would be higher. Moreover, the preinduction optimization based on a passive leg raise and fluid optimization might seem time-consuming, but considering the average length of this procedure, we did not believe the additional time had a great impact on operating room cost or utilization. Another limitation was that the control arm was designed with a fixed algorithm, but we believed that creating an algorithm for the standard arm would minimize the variability in fluid administration still found in different practitioners, allowing us to draw more specific conclusions with a smaller population. Finally, the two arms received different types of fluids; we designed our protocol similarly to the one used by Ramsingh et al8 and decided to maintain the volume expansion with albumin 5% for both arms, one guided by cardiac output monitoring, the other guided by blood loss. The Optimise trial that was published in 2014 used a similar approach.10 A more recent study by Kabon et al21 has looked specifically at the effect of colloid vs crystalloid in a goal-directed fluid therapy setting and found no difference in either complications or toxicity, which confirms the safety of this approach.

In conclusion, we did not find a statistically important benefit in postoperative ileus or 30-day perioperative outcomes for individualized goal-directed fluid therapy versus a predefined standard fluid management in a highly comorbid population undergoing radical cystectomy for bladder cancer. Future studies should be developed using standardized definitions of outcomes and tight delineation of patient populations.

Supplementary Material

Supplemental

Acknowledgments

Research Support: This research was supported by the Sidney Kimmel Center for Prostate and Urologic Cancers and funded in part by the NIH/NCI Cancer Center Support Grant to Memorial Sloan Kettering Cancer Center (P30 CA008748), and by the Gary Gladstein Family. None of the funding sources had any role in study design, data collection, data analysis, data interpretation, the writing of the report, or the decision to submit the paper for publication.

Footnotes

Competing Interests: Dr. Fischer has undertaken paid consulting work for Edwards Lifesciences. The remaining authors declare no competing interests.

REFERENCES

  • 1.Giglio MT, Marucci M, Testini M, Brienza N: Goal-directed haemodynamic therapy and gastrointestinal complications in major surgery: a meta-analysis of randomized controlled trials. Br J Anaesth 2009; 103:637–646 [DOI] [PubMed] [Google Scholar]
  • 2.Vather R, Trivedi S, Bissett I: Defining postoperative ileus: results of a systematic review and global survey. J Gastrointest Surg 2013; 17:962–972 [DOI] [PubMed] [Google Scholar]
  • 3.Froehner M, Brausi MA, Herr HW, Muto G, Studer UE: Complications following radical cystectomy for bladder cancer in the elderly. Eur Urol 2009; 56:443–454 [DOI] [PubMed] [Google Scholar]
  • 4.Svatek RS, Fisher MB, Williams MB, Matin SF, Kamat AM, Grossman HB, Nogueras-González GM, Urbauer DL, Dinney CP: Age and body mass index are independent risk factors for the development of postoperative paralytic ileus after radical cystectomy. Urology 2010; 76:1419–1424 [DOI] [PubMed] [Google Scholar]
  • 5.Shabsigh A, Korets R, Vora KC, Brooks CM, Cronin AM, Savage C, Raj G, Bochner BH, Dalbagni G, Herr HW, Donat SM: Defining early morbidity of radical cystectomy for patients with bladder cancer using a standardized reporting methodology. Eur Urol 2009; 55:164–174 [DOI] [PubMed] [Google Scholar]
  • 6.Gan TJ, Soppitt A, Maroof M, el-Moalem H, Robertson KM, Moretti E, Dwane P, Glass PS: Goal-directed intraoperative fluid administration reduces length of hospital stay after major surgery. Anesthesiology 2002; 97:820–826 [DOI] [PubMed] [Google Scholar]
  • 7.Cannesson M, Ramsingh D, Rinehart J, Demirjian A, Vu T, Vakharia S, Imagawa D, Yu Z, Greenfield S, Kain Z: Perioperative goal-directed therapy and postoperative outcomes in patients undergoing high-risk abdominal surgery: a historical-prospective, comparative effectiveness study. Crit Care 2015; 19:261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Ramsingh DS, Sanghvi C, Gamboa J, Cannesson M, Applegate RL 2nd: Outcome impact of goal directed fluid therapy during high risk abdominal surgery in low to moderate risk patients: a randomized controlled trial. J Clin Monit Comput 2013; 27:249–257 [DOI] [PubMed] [Google Scholar]
  • 9.Calvo-Vecino JM, Ripollés-Melchor J, Mythen MG, Casans-Francés R, Balik A, Artacho JP, Martínez-Hurtado E, Serrano Romero A, Fernández Pérez C, Asuero de Lis S; FEDORA Trial Investigators Group: Effect of goal-directed haemodynamic therapy on postoperative complications in low-moderate risk surgical patients: a multicentre randomised controlled trial (FEDORA trial). Br J Anaesth 2018; 120:734–744 [DOI] [PubMed] [Google Scholar]
  • 10.Pearse RM, Harrison DA, MacDonald N, Gillies MA, Blunt M, Ackland G, Grocott MP, Ahern A, Griggs K, Scott R, Hinds C, Rowan K; OPTIMISE Study Group: Effect of a perioperative, cardiac output-guided hemodynamic therapy algorithm on outcomes following major gastrointestinal surgery: a randomized clinical trial and systematic review. JAMA 2014; 311:2181–2190 [DOI] [PubMed] [Google Scholar]
  • 11.Myles PS, Bellomo R, Corcoran T, Forbes A, Peyton P, Story D, Christophi C, Leslie K, McGuinness S, Parke R, Serpell J, Chan MTV, Painter T, McCluskey S, Minto G, Wallace S; Australian and New Zealand College of Anaesthetists Clinical Trials Network and the Australian and New Zealand Intensive Care Society Clinical Trials Group: Restrictive versus liberal fluid therapy for major abdominal surgery. N Engl J Med 2018; 378:2263–2274 [DOI] [PubMed] [Google Scholar]
  • 12.Pillai P, McEleavy I, Gaughan M, Snowden C, Nesbitt I, Durkan G, Johnson M, Cosgrove J, Thorpe A: A double-blind randomized controlled clinical trial to assess the effect of Doppler optimized intraoperative fluid management on outcome following radical cystectomy. J Urol 2011; 186:2201–2206 [DOI] [PubMed] [Google Scholar]
  • 13.Wuethrich PY, Burkhard FC, Thalmann GN, Stueber F, Studer UE: Restrictive deferred hydration combined with preemptive norepinephrine infusion during radical cystectomy reduces postoperative complications and hospitalization time: A randomized clinical trial. Anesthesiology 2014; 120:365–377 [DOI] [PubMed] [Google Scholar]
  • 14.Clavien PA, Sanabria JR, Strasberg SM: Proposed classification of complications of surgery with examples of utility in cholecystectomy. Surgery 1992; 111:518–526 [PubMed] [Google Scholar]
  • 15.Gómez-Izquierdo JC, Trainito A, Mirzakandov D, Stein BL, Liberman S, Charlebois P, Pecorelli N, Feldman LS, Carli F, Baldini G: Goal-directed fluid therapy does not reduce primary postoperative ileus after elective laparoscopic colorectal surgery: a randomized controlled trial. Anesthesiology 2017; 127:36–49 [DOI] [PubMed] [Google Scholar]
  • 16.American Kidney Fund. Stages of chronic kidney disease (CKD). https://www.kidneyfund.org/kidney-disease/chronic-kidney-disease-ckd/stages-of-chronic-kidney-disease/. Accessed on November 21, 2019
  • 17.Winer AG, Sfakianos JP, Puttanniah VG, Bochner BH: Comparison of perioperative outcomes for epidural versus intravenous patient-controlled analgesia after radical cystectomy. Reg Anesth Pain Med 2015; 40:239–244 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Donat SM: Standards for surgical complication reporting in urologic oncology: time for a change. Urology 2007; 69:221–225 [DOI] [PubMed] [Google Scholar]
  • 19.Cannesson M, Musard H, Desebbe O, Boucau C, Simon R, Hénaine R, Lehot JJ: The ability of stroke volume variations obtained with Vigileo/FloTrac system to monitor fluid responsiveness in mechanically ventilated patients. Anesth Analg 2009; 108:513–517. [DOI] [PubMed] [Google Scholar]
  • 20.Wuethrich PY, Burkhard FC: Improved perioperative outcome with norepinephrine and a restrictive fluid administration during open radical cystectomy and urinary diversion. Urol Oncol 2015; 33:66.e21–24 [DOI] [PubMed] [Google Scholar]
  • 21.Kabon B, Sessler DI, Kurz A, Crystalloid–Colloid Study Team: Effect of intraoperative goal-directed balanced crystalloid versus colloid administration on major postoperative morbidity: a randomized trial. Anesthesiology 2019; 130:728–744 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental

RESOURCES