Evaluation of a risk score for interhospital transport of critically ill patients

Abstract

Background

Interhospital transfer imposes essential risk for critically ill patients. The Risk Score for Transport Patients (RSTP) scale can be used as a triage tool for patient severity.

Methods

In total, 128 transfers of critically ill patients were classified in two groups of severity according to the RSTP. Statistical analysis was performed using the receiver operating characteristic (ROC) curve and goodness of fit statistics.

Results

In total, 66 patients (51.5%) were classified as group I and 62 (48.4%) as group II. Major en route complications were more common in group II patients (19.3% v 3%, p<0.001). Haemodynamic instability was the most common complication. There were significant differences in the mean risk scores between group I and II patients (mean (SD) 4.48 (1.01) v 11.04 (3.47), p<0.001). Discrimination power of RSTP was acceptable (area under the ROC curve 0.743; cutoff value ⩾8). Goodness of fit was adequate (p = 0.390).

Conclusion

The RSTP had acceptable discrimination and adequate goodness of fit and could be considered as a triage tool. Haemodynamic instability is the most common problem encountered during transfer.

Interhospital transfer of critically ill patients is common, even in countries with well developed health care systems.¹ Patients with unresolved, evolving, or incompletely evaluated life threatening conditions are transferred to other hospitals, often under substandard care.¹,² The main reasons for interhospital transfers are lack of intensive care unit (ICU) beds, specialists, diagnostic tools, and therapeutic means.³,⁴ Interhospital transfer of critically ill patients imposes essential risks for both patients and personnel, thus a well established risk to benefit ratio should exist to justify the procedure.¹,⁵,⁶,⁷,⁸ Whichever system of transport is followed, a well validated scale of risk of transport is needed for the best selection of the means and personnel to be used.⁵,⁹,¹⁰

In Greece, a hospital based team of doctors, mainly from the anesthesia department, carries out interhospital ground transports of critically ill patients. In our hospital, the lack of qualified emergency physicians forced the ICU medical staff and the anesthetists to assume this task. The objective of our study was to validate the performance of the recently proposed Risk Score for Transport Patients (RSTP; tables 1–3), in predicting patients most likely to develop major complications during interhospital transport, and deciding on the most appropriate means of transport.⁹

Table 2 Appendix A to table 1: medication for risk groups .

Group I

Inotropics

Vasodilators

Antiarrhythmics

Bicarbonate

Analgesics

Antiepileptics

Steroids

Manitol 20%

Trombolytics

Naloxone

Thoracic tube

Suction

Group II

Inotropics + vasodilators

MAST.

Infant incubator

General anaesthetics

Uterine relaxants

Table 3 Appendix A to table 1: management of at risk patients .

Points		Group		Vehicle		Staff
0–2		0		Conventional ambulance		None
3–6		I		Conventional ambulance		Nurse
Over 6		II		Group ICU		Doctor + nurse

Table 1 Risk Score for Transport Patients* .

Measurement		Score
1. Haemodynamics
Stable		0
Moderately stable (requires volume <15 ml/min in adults)		1
Unstable (requires volume >15 ml/min or inotropics or blood)		2
2. Arrhythmias (existing or probable)
No
Yes, not serious (and AMI after 48 hours)		1
Serious (and AMI in the first 48 hours)		2
3. ECG monitoring
No		0
Yes (desirable)		1
Yes (essential)		2
4. Intravenous line
No		0
Yes		1
Pulmonary artery catheter		2
5. Provisional pacemaker
No		0
Yes (not invasive). Always AMI in the first 48 hours		1
Yes (endocavity)		2
6. Respiration
Respiratory rate between 10 and 14 breaths/min in adults		0
Respiratory rate between 15–35 breaths/min in adults		1
Apnoea <10 or >36 or irregular breathing		2
7. Airway
No		0
Yes (Guedel tube)		1
Yes (intubation or tracheostomy)		2
8. Respiratory support
No		0
Yes (oxygen therapy)		1
Yes (mechanical ventilation)		2
9. Assessment
GCS  = 15		0
GCS 8–14		1
GCS <8 and/or neurological disorder		2
10. Prematurity
Newborn >2000 g		0
Newborn between 1200 and 2000 g		1
Newborn <1200 g		2
11. Technopharmacological support (actual or en route)
None		0
Group I		1
Group II		2

*Adapted from Etxebarria MJ et al.⁹

PATIENTS AND METHODS

Our hospital is a 230 bed community hospital with seven ICU/critical care unit (CCU) beds. It provides health care for some 100 000 people (the number increases significantly during the summer months). Our ICU can provide intensive care to all critically ill adult patients except for those in need of neurosurgical or cardiothoracic intervention or paediatric ICU. These patients after initial stabilisation and evaluation are transferred to the University Hospital of Heraklion, 80 km away (mean of 60 minutes' driving time, range 55–65 minutes).

All critically ill patients transported to University Hospital, Heraklion during a 1 year period (1 November 2001 to 31 October 2002) were prospectively included in our study. Recommendations for uniform reporting of data following major trauma were followed; the Utstein style was used to state patients with multitrauma.¹¹ After initial evaluation and stabilisation, patients in need for transport were classified into two groups of severity using RSTP. Group I comprised patients scoring <7 points and group II comprised patients scoring ⩾7 points.

Patients were transported by a surface ambulance (a mobile ICU) equipped with all the necessary means for continuous advanced life support. The transport team for group I patients consisted of a doctor qualified in advanced life support (usually a trainee in ICU) and two paramedics. Group II patients were escorted by a qualified intensive care physician, or an anaesthetist in the case of children or neonates. Patients >12 years old were considered as adults.

Death during transport, major en route complications, ICU admission, and the ICU mortality rate of transferred patients were recorded. En route complications were defined as minor if they could be solved without any invasive intervention or additional pharmacological treatment (other than sedation).

Major complications were defined as: cardiac or respiratory arrest, decrease of >3 points in Glasgow Coma Score (GCS), fall of PaO₂/FiO₂ ratio by >50 or decrease in SaO₂ by >5% in adults or >3% in children/neonates, hypertension (systolic blood pressure (SBP) >200 mm Hg), hypotension (SBP ⩽90 mm Hg), life threatening arrhythmias, or en route need of any invasive procedure.

Statistical analysis

Groups were compared by t test for continuous variables and by χ² test for categorical data. Descriptive statistics were expressed as mean (SD) unless otherwise stated. Statistical significance was set at p = 0.05. The discrimination of the RSTP was tested using the receiver operating characteristic (ROC) curve and area under the curve (AUC).¹² This curve is a plot of sensitivity versus 100% specificity. Accuracy can be obtained by calculating the AUC. An accuracy of 1.0 is optimum and an area of 0.5 indicates no discriminatory capacity in the entire test As previously suggested, we aimed to classify AUC between 0.7 and 0.8 as “acceptable” and between 0.8 and 0.9 as “excellent” discrimination.¹³ The prognostic performance of the RSTP was analysed by sensitivity, specificity, overall correctness of prediction, positive (PPV) and negative predictive values (NPV), and likelihood ratio of positive test (LRPT).¹⁴ The LRPT is a function of sensitivity and specificity according to the following formula: sensitivity/(100%−specificity). In other words, the LRPT indicates the extent to which a positive test result will increase the probability of the event under consideration. Likelihood ratios of 1–2 alter probability to a small and rarely important degree, whereas those of 2 –5 generate small but sometimes important change in probability. Likelihood ratios of 5–10 and those >10 generate moderate and large shifts from pre‐test probability respectively. All the above variables were calculated both at the cutoff point originally suggested (⩾ than 7), and at the cutoff value giving the best Youden index.⁹,¹⁵ Calibration of the RSTP was assessed using the Hosmer‐Lemeshow goodness of fit statistic, which divides subjects into deciles based on predicted probabilities of death and then computes a χ² from observed and expected frequencies.¹⁶ Lower χ² values and higher p values are associated with a better fit. A good fit was defined as p>0.05.

RESULTS

In total, 128 patients (91 male and 37 female patients; median age 48.5 years, range 0–92) were prospectively enrolled in the study. Most of the transports (54%) occurred during non‐working hours (between 1600 and 0800 on working days) or during weekends. The mean transport time was 58 minutes (range 50–70).

Of these, 66 patients (51.5%) were classified as group I (43 adults, 12 children. and 11 neonates), and 62 patients (48.4%) as group II (53 adults, 4 children. and 5 neonates). Underlying diseases and grouping of patients with regard to RSTP are listed in table 4.

Table 4 Underlying diseases of patients subjected to interhospital transfer.

		Group I		Group II
Adults
Head trauma/trauma		20 (30.3%)		24 (38.7%)
CVA*		5 (7.57%)		6 (9.67%)
Cardiac disease		6 (9.09%)		16 (25.8%)
Respiratory disease		4 (6.06%)		3 (4.83%)
Other†		8 (12.12%)		4 (6.45%)
Children
Head trauma		5 (7.57%)		1 (1.61%)
Infections		5 (7.57%)		1 (1.61%)
Other†		2 (3.03%)		2 (3.22%)
Neonates
Prematures		10 (15.14%)		4 (6.44%)
Other†		1 (1.51%)		1 (1.61%)

*CVA, cerebrovascular accident; †other includes pancreatitis, upper gastrointestinal tract bleeding, burns, neurological, surgical, and obstetric causes.

The mean (SD) age of group II adult patients was younger than that of group I adult patients (57.9 (20.6) v 50.2(22.1); p 0.041). The mean RSTP for group II patients was significantly higher than that of group I patients (11 (3.4) v 4.8 (1); p <0.001).

Invasive intervention (placement of a temporary intravenous pacemaker placement) prior to transport was considered necessary for five group I patients. In contrast, all group II patients had arterial and central venous lines in situ prior to transfer, 42 (71.6%) were receiving inotropes, and 45 (72.5%) were intubated.

No deaths occurred during transfer. Two major complications (hypertensive crisis in patient 1 and aspiration in patient 2; table 5) were encountered during the transfer of group I patients.

Table 5 Major complications during interhospital transfer.

Patient no.		Disease		Complication		Age (years)		Sex		Score		Survival
1		CVA*		Hypertension		72		F		5		Yes
2		Premature		Aspiration		0		F		4		Yes
3		Trauma		Shock		20		M		12		Yes
4		Trauma		Shock		19		M		16		Yes
5		Trauma		Shock		75		F		17		No
6		Trauma		Shock		35		M		12		Yes
7		Head trauma*		Shock		36		F		8		Yes
8		Head trauma		Coma		18		M		12		Yes
9		Head trauma*		Shock		19		M		11		Yes
10		Trauma		Shock		56		M		7		No
11		AMI		Shock		65		M		9		Yes
12		AMI		APO		58		M		9		No
13		CVA		Hypertension		59		M		12		Yes
14		CVA		Hypertension		60		M		15		Yes

Numbers in the first columns represent consecutive patients. CVA, cerebrovascular accident; AMI, acute myocardial infarction; APO, acute pulmonary oedema; *Cases 7 and 9 were mutitrauma patients.

One patient died in the operating room during the following 24 hours; however, his death was considered to be unrelated to transfer. Of 66 group I patients, 16 (24.2%) were admitted to ICU/CCU at the receiving hospital during the subsequent 24 hours following transfer. Four died in ICU (overall group I mortality 7.5%; ICU mortality of group I patients 25%). In contrast, 12 group II patients (19.3%) developed major en route complications (p<0.001 compared with group I patients). Rapid deterioration of consciousness level in a head injured patient (patient number 8; table 5) required tracheal intubation en route. Acute pulmonary oedema in a patient with acute myocardial infarction (patient number 12; table 5) was effectively treated by continuous positive airway pressure application, 100% oxygen administration, and intravenous frusemide. The remaining major problems recorded in group II patients were cardiovascular instability (shock in eight cases, hypertension in two; table 5). Shock was related to trauma in six of eight patients (table 5). All group II patients were directly admitted to the operating room or ICU/CCU of the receiving hospital. Of 62 group II patients, 12 (19.3%) failed to survive ICU.

The ROC curve for RSTP is shown in fig 1. The discrimination power of RSTP AUC was acceptable (AUC 0.743; SE 0.0635; p<0.001; 95% confidence interval 0.619 to 0.868). Table 6 shows predictive values of the RSTP calculated at the cutoff point originally proposed (⩾7) and at the cutoff point giving the best Youden index (⩾8 in our series).

Figure 1 ROC curve for risk score for transport patients (RSTP).

Table 6 Predictive performance of risk score for transport patients at the cutoff point originally proposed (⩾7) and at the cutoff point giving the best Youden index (⩾8).

		Cutoff ⩾7		Cutoff ⩾8
Sensitivity		78%		71%
Specificity		65%		73%
Correct classification rate		67%		73%
Positive predictive value		22%		25%
Negative predictive value		96%		95%
Youden index		0.44		0.45
LRPT		2.29		2.71
CI of LRPT		1.58 to 3.33		1.72 to 4.26

LRPT, likelihood ratio of positive test; CI, confidence interval.

Table 7 Deciles risk for Risk Score for Transport Patients.

Decile		n		PE		AE		NE		ANE
1		13		0.5		0		12.4		13
2		20		1.0		1		19.0		19
3		21		1.3		1		19.7		20
4		12		0.8		0		11.2		12
5		12		1.0		1		11.0		11
6		15		1.5		3		13.5		12
7		16		2.6		5		13.4		11
8		16		4.1		2		11.9		14
9		3		1.1		1		1.9		2

PE, predicted en route events; AE, actual en route events; NE, no en route events predicted; ANE, actual no en route events.

The Hosmer‐Lemeshow statistic revealed adequate goodness of fit for RSTP (χ² = 7389, df = 7, p = 0.390). Tables of deciles risk are shown in table 5.

Overall mortality was significantly higher in group II than group I patients (7.5% v 19.3%; p = 0.025). Howeve,r when considering mortality rates of group I and II ICU admissions (25% v 19.3%) the difference did not reach statistical significance (p = 0.152). Although the ICU mortality of group II patients who suffered major complications during transport (12 patients, three deaths; mortality 25%) was higher than that of group II patients in whom no major en route complications were recorded (50 patients, nine deaths; mortality 18%) the difference did not reach statistical significance (p = 0.291).

DISCUSSION

Our results confirm those of others that interhospital transport of critically ill patients can be safe if appropriate measures are taken before and during transport.¹,¹⁷,¹⁸,¹⁹ No deaths were encountered during transport. Because severe trauma and complicated myocardial infarctions were the most common causes for transport (table 4), shock was the most reasonably expected problem during transfer (table 5). In most cases, this was not really a complication of transfer but rather the reason for it. In four of five trauma patients with shock (patients 3–6, table 5), the reason was penetrating chest trauma necessitating thoracic surgery unavailable in our hospital. All patients were rapidly transported after chest drainage and endotrachial intubation. Blood and blood substitutes were given en route to keep systolic blood pressure >90 mmHg. The emergency department of the receiving hospital was bypassed in all cases and patients were directed straight to the operating theatre. All patients survived surgery and three of the four survived ICU. Two trauma patients with shock had rapidly expanding epidural haematomas and liver contusions (patients 7 and 9, table 5). They were rapidly transported in the same manner as the previous cases. They also survived the operation and ICU stay. Finally, one patient in cardiogenic shock (as a result of acute myocardial infarction) was transferred for emergency coronary balloon angioplasty.

Shock is difficult to define, evaluate, and treat, and although transported patients must be physiologically stable before transfer, postponing definite care in the presence of an evolving disease such as head trauma or acute coronary syndrome can be fatal or a substantial risk factor for multiple organ failure and intensive care mortality.⁵ Our practice of rapid transfer under full circulatory support, the strict control by a senior ICU doctor during transfer, the absolute cooperation with the receiving hospital, and the relatively short transfer time resulted in the patients' good outcome.

In contrast to difficulties and arguments about cardiovascular stabilisation before transfer, respiratory support strategies are uniformly accepted and effective in preventing complications en route.²⁰ The adopted practice by our team to intubate before transport all patients suffering from or susceptible to respiratory or CNS deterioration during the trip (72.5% in our group II patients) and the advanced technology of modern portable mechanical ventilators, practically solved the respiratory part of cardiorespiratory support during transfer. Novel therapies for severe oxygenation failure (from ventilation in prone position to extracorporeal membrane oxygenation) have recently made transport over long distances achievable even for patients with severe oxygenation failure.¹⁷,¹⁸

Evidence supporting the high quality of the care provided to our patients during transfer (effective management of en route complications), is the non‐significant difference between the ICU mortality of those patients who suffered major en route complications versus those who did not. In addition, although not mutually matched in pairs of similar severity, the average mortality rate of transferred patients who required admission to the ICU of the receiving hospital (78 patients, 18 deaths, 23%), was not significantly different from the average mortality of the other patients of the receiving ICU (20%; unpublished data). Similar results have been reported by Surgenor et al, who did not find any difference in ICU mortality between patients admitted to ICU after referral from other hospitals and patients admitted to the same ICU from the wards of a tertiary receiving hospital.²¹

The utility of using risk scores for transport patients is controversial, as their principal aim is to limit the number of staff used in transfers. Most of our transfers took place during non‐working hours, when specialist transport teams were rarely available. Therefore, patient triage at the time of referral is important. Different indexes of risk of transport have been tested, none of them being widely accepted.⁵,⁹,²²,²³ Some authors believe that high quality care during transport will abolish the relationship between severity of illness and risk of complications, and paradoxically, scoring systems may therefore only be of value in predicting the risk of inexpert transfer.²³

In this study, the RSTP had acceptable discrimination value and adequate goodness‐of‐fit in predicting major en route complications. Increasing the cutoff value (from ⩾7 to ⩾8) further increased specificity, correct classification rate, PPV, and LRPT, although at the cost of some small loss in sensitivity. The ROC curves provide a complete view of the accuracy of the prognostic scores under study and permitdirect comparison of the AUC obtained for various prognostic scores.¹²,¹³ Although it can be stated that AUC and overall accuracy of prediction are less useful tools for evaluation of a risk score in critically ill patients (because when transferring such patients, missing one severe en route complication is potentially more disastrous than correctly predicting the majority of the more numerous transfers without complications), we strongly believe that its high negative predictive value precludes the aforementioned argument. Finally, the RSTP had also a LRPT high enough to generate as previously mentioned small but sometimes important changes in probability.

This study has the major result that RSTP has a correlation with the outcome of the patients as well as the complications observed during the transport. Despite initial stabilisation and meticulous follow up during transport, group II patients were more severely ill than group I patients, thus they had a higher RSTP. It seems that if major en route complications are effectively and on time managed, the prognosis depends not on the 60 minute transfer delay but on the severity of the underlying condition, which is properly reflected on the RSTP. However, our primary objective was to validate the performance of the RSTP in predicting patients most likely to develop major complications during interhospital transport. Thus, the performance of RSTP in predicting long term mortality was not validated in terms of discrimination and calibration.

CONCLUSIONS

Interhospital transport of critically ill patients can be safe in the hands of experts. Shock remains the most common problem encountered during transport. The RSTP seems effective in differentiating critically ill patients prone to develop major en route complications. Further studies with larger numbers of patients are however required to confirm its applicability and to define its optimal cutoff value.

Abbreviations

AUC - area under the curve

CCU - critical care unit

GCS - Glasgow Coma Score

ICU - intensive care unit

LRPT - likelihood ratio of positive test

NPV - negative predictive value

PPV - Positive predictive value

ROC - receiver operating characteristic

RSTP - Risk Score for Transport Patients

SBP - systolic blood pressure