Assessment of a novel marker of ICU strain, the ICU Activity Index, during the COVID-19 pandemic in Victoria, Australia

Abstract

Objectives: To validate a real-time Intensive Care Unit (ICU) Activity Index as a marker of ICU strain from daily data available from the Critical Health Resource Information System (CHRIS), and to investigate the association between this Index and the need to transfer critically ill patients during the coronavirus disease 2019 (COVID-19) pandemic in Victoria, Australia.

Design: Retrospective observational cohort study.

Setting: All 45 hospitals with an ICU in Victoria, Australia.

Participants: Patients in all Victorian ICUs and all critically ill patients transferred between Victorian hospitals from 27 June to 6 September 2020.

Main outcome measure: Acute interhospital transfer of one or more critically ill patients per day from one site to an ICU in another hospital.

Results: 150 patients were transported over 61 days from 29 hospitals (64%). ICU Activity Index scores were higher on days when critical care transfers occurred (median, 1.0 [IQR, 0.4–1.7] v 0.6 [IQR, 0.3–1.2]; P < 0.001). Transfers were more common on days of higher ICU occupancy, higher numbers of ventilated or COVID-19 patients, and when more critical care staff were unavailable. The highest ICU Activity Index scores were observed at hospitals in north-western Melbourne, where the COVID-19 disease burden was greatest. After adjusting for confounding factors, including occupancy and lack of available ICU staff, a rising ICU Activity Index score was associated with an increased risk of a critical care transfer (odds ratio, 4.10; 95% CI, 2.34–7.18; P < 0.001).

Conclusions: The ICU Activity Index appeared to be a valid marker of ICU strain during the COVID-19 pandemic. It may be useful as a real-time clinical indicator of ICU activity and predict the need for redistribution of critical ill patients.

Limited access to, or excess demand for, available intensive care unit (ICU) resources, also known as “ICU strain”, may be associated with adverse patient outcomes.¹ A fully occupied ICU may delay the definitive management of a patient who requires critical care,² or compromise the care of those in ICU. It may result in some patients being denied admission to an ICU.³ Both are associated with increased mortality risk.⁴ Although several individual indices of hospital and ICU strain have been investigated,5, 6 none is ideal, and composite indices to measure strain have rarely been examined.⁶

The coronavirus disease 2019 (COVID-19) pandemic placed a marked increase in demand on health care resources throughout the world, with requirement for ICU services exceeding capacity in many countries, including China,⁷ Italy,⁸ the United Kingdom⁹ and the United States.¹⁰ Lack of access due to the strain on critical care services was associated with adverse patient outcomes.11, 12

In June 2020, in response to rising notifications of COVID-19 within Victoria, Australia,¹³ ICU clinical directors representing each of the nine state health care service clusters commenced a daily morning meeting with representatives from Ambulance Victoria, Safer Care Victoria and the Victorian Department of Health and Human Services. The intention of the daily meeting was to quantify clinical demand, identify and support ICUs under strain and, where necessary, proactively facilitate transfers to another hospital’s ICU to address excess local demand. This situation provided an opportunity to develop and validate a de novo real-time composite measure of ICU strain.

Our hypothesis was that strain within an ICU would be associated with the need to transfer a critically ill patient to another hospital with greater ICU capacity. Our objectives were to measure daily levels of ICU strain using an ICU Activity Index and to assess its association with the acute interhospital transfer of one or more critically ill patients to another hospital’s ICU.

Methods

This project was approved as a low risk project by the Human Research Ethics Committee of The Alfred Hospital (HREC: 480/20).

Data sources

De-identified data on all adult critically ill patients transferred from an ICU or from the emergency department to another hospital’s ICU (in the same state) between 27 June and 6 September 2020 were extracted from the Ambulance Victoria’s Adult Retrieval Victoria Information System. We excluded transfers for specialist (neurosurgical, cardiac or trauma) services not available at the sending hospital, transfers from hospitals from another jurisdiction, and transfers from sending hospitals without an onsite ICU.

Real-time ICU activity data from all Victorian acute care hospitals were extracted from the Critical Health Resource Information System (CHRIS) for the same period. CHRIS is a real-time dashboard of specific ICU activity and acuity and resources, developed and implemented nationally by the Australian and New Zealand Intensive Care Society,¹⁴ Ambulance Victoria, the Australian Government Department of Health, and Telstra Purple in response to the COVID-19 pandemic.

Data extracted from CHRIS were used to calculate the total daily number of available, staffed and equipped (“open available”) ICU beds, the number of additional ICU beds that were open above the baseline reported at the beginning of the study period, and current daily ICU occupancy — based on a staffing minimum of one critical care nurse for every ICU patient and one for every two high dependency (step-down) patients within the ICU. Individual patient length of stay and outcome data were not reported.

The ICU Activity Index

A numerical Activity Index was calculated for each ICU as follows:

$$\text{ICU}\text{Activity}\text{index}=\frac{\text{COVIDs}+\text{MV}+\text{RRT}+\text{ECMO}+\text{ICU}}{\text{Total}\text{daily}\text{open}\text{available}\text{ICU}\text{beds}}$$

where COVIDs = number of COVID-19 patients within the ICU; MV = number of ICU patients receiving invasive ventilation; RRT = number of ICU patients receiving renal replacement therapy; ECMO = number of ICU patients receiving extracorporeal membrane oxygenation; ICU = number of patients requiring 1:1 nurse to patient ratio; and total daily open available ICU beds = total number of open, equipped, 1:1 staffed bed spaces in the ICU declared daily on CHRIS. Thus, the minimum value was “0” if all patients in the ICU were being nursed 1:2, none required renal replacement, invasive ventilation or ECMO and there were no COVID-19 patients. The maximum value was “5” if every patient in the ICU was a ventilated COVID-19 patient on renal replacement therapy and ECMO.

The ICU Activity Index was calculated daily at 8.30 am. If site information had not been updated during the previous 24 hours, the ICU Activity Index was considered as unavailable. There was no imputation of missing data.

Primary outcome

The primary outcome was the acute interhospital transfer of one or more critically ill patients to another hospital ICU during the 24-hour calendar day of the calculated ICU Activity Index. Decisions to transfer patients were based on the clinical judgement of the treating intensive care specialist communicated to their local cluster-lead representative. Transfers included, but were not restricted to, patients with COVID-19.

The ICU Activity Index value was not accessible to treating clinicians. Those with access to the ICU Activity Index were not engaged in the clinical decision to transfer a patient.

Statistical methods

For descriptive purposes, ICUs were categorised into one of three large geographic groups — north-western Melbourne, south-eastern Melbourne and regional Victoria — representing the regions with the highest, intermediate and lowest numbers of community COVID-19 notifications respectively.¹⁵ Each hospital was classified as either a public tertiary, public metropolitan, public rural or private facility.

Data for each ICU site on each day (“site-day”) were reported either as number (proportion) or median (interquartile range [IQR]). Maximum, minimum and mean values were also provided to describe resource use and demand, where appropriate. Statistical comparisons employed χ², Wilcoxon rank sum or Kruskal–Wallis tests depending on type and distribution of data.

Descriptive statistics, kernel density and graphical time series plots were used to show variation in the ICU Activity Index over time. Associations with outcome were investigated using mixed effects hierarchical logistic regression, with hospital site treated as a random intercept, and results reported as odds ratios (ORs) and 95% confidence interval (CI). Akaike and Bayesian information criteria were used for best model selection.¹⁶ Cut points to maximise specificity and sensitivity of the ICU Activity Index were determined using the Youden Index.¹⁷ A two-sided P < 0.05 was considered significant. All data were analysed using Stata version 16.1 (StataCorp, Texas, USA).

Results

All 45 public and private adult hospitals with an onsite ICU within the state of Victoria contributed data. There were 150 ICU patients (1.7%) transported on 61 study days (84.7%), equivalent to 143 site-days (4.4%), from 29 hospitals (64.4%). During this time, there were a total of 468 170 hospital separations, of which 7911 (1.7%) included an ICU admission, and 2697 (0.6%) confirmed COVID-19 hospital separations, of which 247 (9.2%) were admitted to an ICU.

Patterns of the ICU Activity Index varied between hospitals and over time. ICU Activity Index scores were higher at sites on days when critical care transfers occurred (median, 1.0; IQR, 0.4–1.7 compared with days when no transfers occurred (median, 0.6; IQR, 0.3–1.2; P < 0.001). Critical care transfers were more frequent from sites where there was higher occupancy; higher COVID-19 patient numbers in the ICU; higher proportion of patients requiring mechanical ventilation, renal replacement therapy or 1:1 critical care nursing; or when there were more critical care staff unavailable (Table 1).

Table 1.

Characteristics of intensive care units (ICUs) on days (ie, site-days) when critical care transfers occurred compared with days with no transfer of critically ill patients between hospitals

	All 45 ICUs (3240 site-days⁎)		Transfer — No (3097 site-days⁎)		Transfer — Yes (29 ICUs, 143 site-days⁎)		P
	Mean	Median (IQR)	Mean	Median (IQR)	Mean	Median(IQR)
ICU Activity Index†	0.7	0.7 (0.3–1.2)	0.7	0.6 (0.3–1.2)	1.1	1.0 (0.4–1.7)	< 0.001
Site-days with ICU Activity Index > 1.6		288/2609 (11%)		254/2486 (10%)		34/123 (28%)	< 0.001
Site-days with ICU Activity Index < 0.6		1231/2609 (47%)		1197/2486 (48%)		34/123 (28%)	< 0.001
Daily ICU patient workload
Total patients in each ICU	8.1	6 (4–9)	8.1	6 (4–9)	8.8	6 (5–9)	0.026
Ventilated patients	2.6	1 (0–3)	2.6	1 (0–3)	3.6	2 (0–4)	< 0.001
COVID-19 patients	0.6	0 (0–1)	0.6	0 (0–0)	1.3	0 (0–2)	< 0.001
Ventilated COVID-19 patients	0.4	0 (0–0)	0.4	0 (0–0)	0.9	0 (0–1)	< 0.001
Critical care staff unavailable	0.9	0 (0–1)	0.8	0 (0–1)	1.6	0 (0–2)	< 0.001
Ventilated patients, %‡	20%	13% (0–33%)	19%	13% (0–33%)	32%	33% (9–50%)	< 0.001
Renal replacement therapy, %‡	5%	0% (0–7%)	4%	0% (0–7%)		0% (0–13%)	0.004
COVID-19 patients, %‡	5%	0% (0–4%)	5%	0% (0–0%)	14%	0% (0–29%)	< 0.001
Ventilated COVID-19 patients, %‡	3%	0% (0–0%)	3%	0% (0–0%)	10%	0% (0–17%)	< 0.001
Daily ICU resources
Physical ICU beds	19.4	12 (8–20)	19.4	12 (8–20)	19.4	13 (8–18)	0.99
Daily open available ICU beds	9.5	7 (5–11)	9.5	7 (5–11)	9.2	7 (5–10)	0.51
Additional ICU beds over baseline§⁎⁎	–0.7	0 (–2 to 0)	–0.7	0 (–2 to 0)	–0.3	0 (–1 to 0)	0.22
Occupancy	62%	61% (40–83%)	61%	60% (38–83%)	71%	74% (50–94%)	< 0.001
Geographic region¶
North-western Melbourne	864/3240 (27%)		813/3097 (26%)		51/143 (36%)	7/12⁎⁎
South-eastern Melbourne	1224/3240 (38%)		1197/3097 (39%)		27/143 (19%)	9/17⁎⁎	< 0.001
Regional Victoria	1152/3240 (36%)		1087/3097 (35%)		65/143 (45%)	13/16⁎⁎
Hospital type¶
Public tertiary	432/3240 (13%)		417/3097 (13%)		15/143 (10%)	5/6⁎⁎
Public metropolitan	720/3240 (22%)		657/3097 (21%)		63/143 (44%)	9/10⁎⁎	< 0.001
Public rural	792/3240 (24%)		732/3097 (24%)		60/143 (42%)	10/11⁎⁎
Private	1296/3240 (40%)		1291/3097 (42%)		5/143 (4%)	5/18⁎⁎

COVID-19 = coronavirus disease 2019; IQR = interquartile range.

^⁎A “site-day” represents one day per site.

^†The ICU Activity Index was available and could be calculated on 81% (2609/3240) of all site-days (72 days, 45 hospitals).

^‡Expressed as a proportion of open ICU beds at the site on each day.

^§Additional ICU beds over baseline = the number of additional open ICU beds over the baseline reported by the ICU at the beginning of the study period.

^¶All sites had at least one or more days when critical care transfers did not occur. Statistics reported as % (n/total site-days).

^⁎⁎Number of sites.

ICU Activity Index scores were highest at hospitals in north-western Melbourne, the region with the highest burden of COVID-19. The highest proportion (51/864, 5.9%) of site-days with critical care transfers occurred at hospitals in this region, with three hospitals transferring more than one patient per day on seven occasions (Online Appendix, table 2). Patients transferred from these hospitals more commonly had respiratory diagnoses (33/58, 57%), required mechanical ventilation (47/58, 81%) and were more likely to be transferred directly from the ICU (41/58, 71%) (Online Appendix, table 3). An example of the daily ICU Activity Index scores from the four metropolitan hospitals in north-western Melbourne and associated days when critical care transfers occurred is shown in Figure 1, with figures for all hospitals presented in the Online Appendix, figures 1a–1c.

Figure 1.

ICU Activity Index (blue shading) and days when critical care patients were transferred (dots) from metropolitan intensive care units (ICUs) in north-western Melbourne

At higher ICU Activity Index levels, there were more critical care transfers, higher levels of ICU occupancy, greater numbers of patients requiring 1:1 nurse to patient ratios, and more critical care staff unavailable (Figure 2). In keeping with its calculation, the ICU Activity Index increased proportionally with the number of patients receiving mechanical ventilation, renal replacement therapy or extracorporeal membrane oxygenation (Online Appendix, figure 2).

Figure 2.

Proportion of days when a critical care transfer occurred; mean number of unavailable critical care staff per site per day; proportion of intensive care unit (ICU) patients who required 1:1 critical care nursing; and daily mean ICU occupancy plotted against rising ICU Activity Index categories

Overall, the point of maximal combined sensitivity (72%) and specificity (48%) for the prediction of a critical care transfer was at a low ICU Activity Index score of 0.6. A high ICU Activity Index score of 1.6 had a sensitivity of 28% and specificity of 90% for interhospital transfer. This was also the level that maximised overall sensitivity and specificity for public tertiary hospitals (Online Appendix, table 4). Kernel density plots for site-days when critical care transfers occurred compared with those when there was no transfer had greatest separation either below an ICU Activity Index score of 0.6 or above 1.6 (Online Appendix, figure 3). A critical care transfer occurred on 12% of occasions when the ICU Activity Index score was 1.6 or more, compared with 3% of occasions when it was below 0.6 (P < 0.001) (Online Appendix, table 5).

After adjusting for confounding factors, such as hospital type, occupancy and number of unavailable critical care staff, a rising ICU Activity Index was associated with an increased risk of a critical care transfer (OR, 4.10; 95% CI, 2.34–7.18; P < 0.001). Public metropolitan and rural hospitals were more likely to transfer patients than public tertiary or private hospitals. Of note, occupancy was not predictive of critical care transfer in the multivariable analysis (OR, 0.48; 95% CI, 0.14–1.64; P = 0.25) (Table 2). These findings were consistent in multiple sensitivity analyses (Online Appendix, tables 6a–6d).

Table 2.

Hierarchical logistic regression to identify factors associated with transfer of one or more critical care patient each day from Victorian intensive care units (ICUs)

	Odds ratio (95% CI)	P
Univariable analysis
ICU Activity Index	3.34 (2.2–5.05)	< 0.001
Multivariable analysis
ICU Activity Index	4.10 (2.34–7.18)	< 0.001
Occupancy	0.48 (0.14–1.64)	0.25
Critical care staff unavailable⁎	1.00 (0.95–1.05)	0.94
Hospital type†
Public tertiary	1.00‡
Public metropolitan	3.75 (1.38–10.15)	0.009
Public rural	8.26 (2.81–24.3)	< 0.001
Private	0.42 (0.11–1.58)	0.20

^⁎Critical care staff unavailable at each site each day due to COVID-19 exposure or disease.

^†Hospital site entered as a random effect (variance, 0.53; 95% CI, 0.23–1.25); area under receiver operating characteristic, 0.84 (95% CI, 0.81–0.87).

^‡Reference value.

Discussion

Summary of findings

The number of critical care transfers in Victoria (during the second wave of the COVID-19 pandemic in Australia) averaged two per day and affected one in 60 ICU patients. A simple ICU Activity Index based on readily available real-time data was a better predictor of the need for a critical care transfer than ICU occupancy alone. The ICU Activity Index may be a pragmatic indicator of ICU strain.

Comparison to previous literature

The need for a marker which can identify ICUs where patient outcomes are at risk because of increased strain is well documented.¹⁸ Strain on critical care services has been reported from many parts of the world during the COVID-19 pandemic. The lack of access to intensive care beds and ICU strain has been associated with higher case fatality rates for COVID-19 patients.1, 11 Interhospital transport of critically ill patients has been recognised as a patient safety mechanism to redistribute demand to match ICU resources during the pandemic.¹⁹

Due to successful public health measures and the localised nature of the second wave of infections in Victoria,20, 21, 22, 23 the COVID-19 pandemic appeared to result in fewer attributable excess deaths in Australia compared with many countries.²² Despite this, concerning levels of ICU strain were seen at hospitals, particularly in north-western Melbourne, which led to the transfer of patients to ICUs with greater capacity. The ICU Activity Index provided an objective measure of the level of ICU strain.

The ICU Activity Index is a composite metric of both ICU census and aggregate daily patient acuity. Commonly used measures of ICU strain such as high occupancy and daily census data may have variable relationships to patient outcomes.⁶ Large studies have demonstrated either no relationship between high occupancy and adverse patient outcomes²⁴ or only present in a closed medical staffing model with intensive care specialist-led services,²⁵ the model used in Australia and New Zealand. A large study from the UK suggested that the relationship between markers of strain and mortality was strongest when patient acuity was also taken into account.²⁶ The ICU Activity Index may be a more robust marker of ICU strain because it includes a measure of complexity as well as occupancy.

Lower staffing ratios, staff shortages and high levels of procedural activity in addition to high patient turnover have been shown to be associated with adverse patient outcomes.²⁷ In our study, levels of occupancy, acuity and staff unavailability were all higher on days with greater ICU Activity Index values.

Implications of findings

This study supports the potential use of the ICU Activity Index as a marker of strain, at least in Australia and New Zealand. It has content validity: greater values were associated with higher proportions of patients requiring invasive ICU therapies (as expected from its calculation) but no one factor was more influential than others. The ICU Activity Index also had face validity: higher values were seen in ICUs with greater occupancy and nursing requirements, and in ICUs that were known to be under strain with high numbers of COVID-19 admissions or high levels of staff unavailability. In addition, the ICU Activity Index had greater predictive validity than other potential markers of strain, such as occupancy and staff shortage, for determining the need to transfer a critically ill patient.

Strengths and limitations

The strengths of the study include engagement with senior clinicians, high compliance of clinical staff with CHRIS data entry, and the quality of the data entered. These clinicians were aware that ICU demand and capacity were being monitored but were unaware of the ICU Activity Index value itself, so it is unlikely to have directly influenced clinician decision making. The ICU Activity Index has now been incorporated into the CHRIS dashboard.

The limitations of the study included absence of clinical data on individual patient characteristics. Patient outcomes were not available. No information was available on metrics such as after-hours or premature discharge from ICU, nor on overall staffing within each ICU. All hospitals were required by the state health department to reduce elective surgery during the study period. It was not possible to determine which transfers were triggered by rising COVID-19 presentations or non-COVID-19 demand, nor how many of the patients transferred were positive for COVID-19 infection. The value of the ICU Activity Index in other states of Australia has not been proven, nor has it been tested during periods of low COVID-19 prevalence.

Future studies

The potential of the ICU Activity Index to quantify and predict ICU strain in real time and act as a trigger for clinical intervention is attractive and warrants further prospective evaluation. Future studies should determine whether there is a relationship between the ICU Activity Index and the capacity to accept critical care transfers, in addition to investigating a direct effect on patient outcomes such length of stay, short and long term survival, and serious adverse events in the ICU population, both during seasonal fluctuations in demand as well as during a pandemic. The potential of the ICU Activity Index for monitoring system responses to changes in public health policy, process implementation and provision of intensive care resources throughout Australia and New Zealand warrants investigation. The Index may also have a role in routine monitoring of ICU capacity in response to public health emergencies such as mass casualty events, bushfires,²⁸ thunderstorm asthma,²⁹ or peaks in seasonal demand.

Conclusion

The ICU Activity Index appeared to be a valid marker of ICU strain and predictive of the transfer of critically ill patients during second wave of the COVID-19 pandemic in Victoria. It is an immediately available measure that can routinely monitor ICU demand and capacity in Australia and New Zealand.

Competing interests

No relevant disclosures.