Surgical pleth index: prospective validation of the score to predict moderate-to-severe postoperative pain

Abstract

Background

Reported outcome benefits after surgical pleth index (SPI, GE Healthcare, Helsinki, Finland) guided anaesthesia are conflicting. One potential explanation may be the lack of evidence for the selection of meaningful SPI target values. A recently published trial found an SPI cut-off of 30 as a ‘best-fit’ to predict moderate-to-severe acute postoperative pain. This prospective trial was designed to validate this target and to investigate the influence of patient age on SPI in this context.

Methods

After ethics approval, 200 patients undergoing non-emergency surgery were enrolled. Data related to SPI, heart rate (HR), and mean arterial pressure (MAP) were recorded for the last 5 min of surgery, just before arousal. After admission to recovery, pain scores (numeric rating scale [NRS], 0–10) were obtained every 5 min for 15 min.

Results

The data of 196 patients were analysed. Receiver-operating curve analysis showed a cut-off SPI value of 29 to be the optimum intraoperative target to discriminate between NRS 0–3 and 4–10. This confirms the ‘best fit’ cut-off for SPI published previously. Though still superior to HR and MAP, the sensitivity and specificity of the SPI were only poor. Age had no influence on the predictive accuracy of SPI.

Conclusions

An SPI of approximately 30 was confirmed as having the best sensitivity/specificity to predict moderate-to-severe pain in the postanaesthesia care unit. However, the predictive accuracy was overall poor and not influenced by patient age.

Clinical trial registration

ACTRN12617001475336.

Editor's key points.

• •Objective measurement of pain in the perioperative period may be used to guide intraoperative analgesia and reduce postoperative pain.
• •One approach uses the surgical pleth index (SPI) based on the autonomic response to a nociceptive stimulus.
• •This study validated a cut-off point of 30 as discriminating between mild pain and moderate to severe pain.
• •There are limitations with SPI sensitivity and specificity that need to be addressed to improve its clinical potential.

The surgical pleth index (SPI, GE Healthcare, Helsinki, Finland) is a dimensionless score which is based on the photoplethysmographic analysis of the pulse wave and the heart beat interval.¹ SPI scores monitored during surgery may reflect a patient’s autonomic response to certain nociceptive stimuli.2, 3, 4 Several studies have since investigated the potential benefits of SPI-guided anaesthesia,5, 6, 7, 8 however with conflicting results. A more recent study by Ledowski and colleagues⁹ hypothesised that an incorrectly chosen target value for the SPI may have contributed to these findings. The same study reported an SPI cut-off value of 30 measured before arousal at the end of surgery to have an optimum sensitivity and specificity for the prediction of moderate-to-severe acute postoperative pain (positive predictive value [PPV] 50% and negative predictive value [NPV] 89.7%, respectively).

However, the results in this study were calculated by means of a post hoc definition of the ‘best-fit’ SPI target, thus requiring further validation.

It was the aim of the current investigation to prospectively test the SPI target of 30 for the prediction of moderate-to-severe postoperative pain. As a previous study¹⁰ described a significant influence of age on the parameters of HR variability, at least in part, the basis for the calculation of SPI, we also aimed to investigate the influence of age on the tested SPI target.

Methods

The trial was approved by the institutional review board (Ethics Committee of the South Metropolitan Health Service, Perth, Australia, protocol RGS523 from September 26, 2017), which also granted a waiver of consent because of the purely observational character of the study. The project was registered with the Australian and New Zealand Clinical Trials Registry (ACTRN12617001475336).

A total of 200 patients were enrolled in this prospective observational trial. In order to accurately validate the results from a pilot trial previously published by us,⁹ the methodology for the current study was kept almost identical to the previous one.

Patients aged 18–95 yr undergoing non-emergency anaesthesia with sevoflurane-fentanyl (tracheal tube or laryngeal mask airway) were included. Exclusion criteria included an age <18 yr, severe peripheral or cardiac neuropathy, significant arrhythmia (i.e. atrial fibrillation or atrio-ventricular block >1st degree), pacemaker, treatment (infusion or bolus) with vasoactive medication during and within 10 min of the data acquisition interval, and any intraoperative treatment with ketamine, beta-receptor blockers, clonidine, beta-receptor agonists, or any other drug suspected to interact with the sympathovagal balance. Excluded were also patients having neuraxial anaesthesia, and patients requiring surgery with a tourniquet (unless already deflated at the start of the intraoperative data acquisition interval [see below]) as the tourniquet-related pain would have potentially significantly influenced the recorded SPI values. Data acquisition was also not performed during phases of patients being either in steep (>10°) Trendelenburg or the anti-Trendelenburg position.

The study commenced after induction of anaesthesia (midazolam, propofol, fentanyl) and at least 5 min before the anticipated end of surgery. In addition to standard anaesthesia monitoring (S5 monitor, GE Healthcare, Helsinki, Finland), SPI was assessed every minute. Anaesthesia was maintained with sevoflurane/fentanyl and, if monitored, state entropy (GE Healthcare) was kept between 40 and 60 during the entire surgery. In the patients in whom state entropy was not monitored (the choice of the attending anaesthetist), the end-expiratory concentration of sevoflurane was kept between 0.8 and 1.3 MAC. Some 5 min to the end of surgery (=skin closure or at the time of wound dressing), SPI and HR were recorded every minute, and the MAP every 2.5 min.

All patients were admitted to the recovery room. Once able to communicate, the patients were asked to quantify their level of pain on a 0–10-point numeric rating scale (NRS) every 5 min for 15 min. Pain treatment was guided by our standard recovery room protocol (fentanyl 20 μg each time pain NRS>3 until a maximum of 100 μg [requiring anaesthetist review]) and not influenced by the study. The maximum pain score from the three scores assessed during the observation interval was used for further analysis.

Statistical analysis

The sample size calculation of n=200 patients was based on the desire to be able to prove a significant difference between an area under the curve (AUC) (receiver operating characteristic [ROC] curve, calculated for the SPI cut-off of 30 to predict moderate-to-severe postoperative pain) of at least 0.7 (as reported in Ledowski and colleagues⁹) from an AUC of 0.5 (no predictive value at all) with an alpha error of 5% and a power of 90%. For this purpose, 50 patients were deemed to be necessary in a no-to-mild pain group and a moderate-to-severe pain group. However, as per our experience with previous studies, the ratio of these two groups could vary between 1:1 and 1:3 (observational study with no influence on actual pain outcomes), and we aimed to include a total of 200 patients.

Data were tested for normal distribution using the Kolmogorov-Smirnov test. Categorical data were compared using the χ² test or Fishers’ exact test, as appropriate. Continuous data were compared using analysis of variance. The ‘best-fit’ cut-off values for SPI, HR, and MAP to predict moderate-to-severe pain in the recovery room were calculated via ROC. The ‘best-fit’ cut-off for each parameter was defined as the value with the highest combined sensitivity and specificity (Youden’s index) for the prediction of moderate-to-severe pain. The influence of age on the predictive value of the SPI was calculated using a logistic regression model. Data are provided as mean (standard deviation) and 95% confidence intervals (CI), as appropriate. The statistical software used was IBM SPSS Statistics version 24 (IBM, Armonk, NY, USA).

Results

During the period from November 2017 until May 2018, 200 consecutive patients meeting the inclusion criteria were included in the trial (no withdrawal of patients as exempt from the need to consent). Data of 196 patients (84 female/112 male; 44 [18] aged 18–87 yr) were analysed. Protocol violation (administration of vasoactive drugs during or too close to the SPI observation period) caused the exclusion of four datasets. The operations were performed by general (77), plastic (38), orthopaedic (40), urology (11), and other (30) surgical specialties.

In the PACU, an average of 7.8 (6.5) min was noted between patient admission and patients’ ability to communicate their pain on an NRS. Over the 15 min observation time, three pain ratings were recorded. Patients reported their highest pain scores as either no (NRS 0; n=54), mild (NRS 1–3; n=26), moderate (NRS 4–5; n=36), or severe pain (NRS 6–10; n=80 [NRS 6=27; NRS 7=26; NRS 8=11; NRS 9=8; NRS 10=8]).

SPI scores measured before arousal at the end of surgery were significantly higher for patients with postoperative moderate-to-severe pain (NRS 4–10) vs no or mild pain (NRS 0–3) (mean [standard deviation]); 95% CI: 30.5 (13.5 [27.8–33.3]) vs 34.7 (14 [31.9–37.6]); P=0.035. The correlation between SPI and the postoperative pain scores per se (NRS 0–10) was, however, not significant.

ROC to distinguish between NRS 0–3 and 4–10 were investigated before arousal at the end of surgery (= last 5 min of surgery) for the mean values of SPI, HR, and MAP. The only significant relationship between one of the investigated parameters and postoperative pain was found for SPI (AUC 0.601; 95% CI 0.52–0.69; P=0.015).

Calculated post hoc for the current dataset, SPI 29 had the highest combined sensitivity (68%) and specificity (57%) to distinguish between patients with/without moderate-to-severe pain in the PACU.

The SPI of 30 (to be tested prospectively as post hoc calculated in a previous study=primary endpoint of this trial) had a sensitivity of 61% and a specificity of 58% (AUC 0.59; 95% CI 0.51–0.67; P=0.036; Fig. 1). The NPV of SPI <30 for the absence of moderate-to-severe pain was 58% (95% CI: 51–64%). The PPV for SPI >30 was 60% (95% CI: 53–67%; Fig. 2).

Fig 1.

Receiver operating characteristics for surgical pleth index (SPI), HR, and MAP to distinguish between no-to-mild (0–3) vs moderate-to-severe (4–10) acute postoperative pain on a numeric rating scale (NRS, 0–10).

Fig 2.

Scatterplot of the highest surgical pleth index (SPI) within the last 5 min of surgery and the highest pain score within 15 min from admission to the recovery room, grouped in no (numeric rating scale [NRS] 0), mild (NRS 1–3), moderate (NRS 4–5), and severe (NRS 6–10). SPI=30 was tested prospectively to predict moderate-to-severe postoperative pain (dotted line).

There was no significant interaction between the age of the patients and their respective pain scores in the PACU, and no significant statistical interaction between age and the accuracy of the SPI to predict patients with higher postoperative pain scores (see supplementary table).

Discussion

Our study found that, although the SPI at the end of surgery was significantly higher in patients later experiencing moderate-to-severe postoperative pain in the PACU, the overall sensitivity and specificity of the parameter for the prediction of pain was relatively low. We were unable to confirm the high PPV (89%) of SPI >30 for moderate-to-severe postoperative pain reported in our previous publication.⁹ However, the previously published results were calculated post hoc (‘best-fit’ SPI cut-off) from a limited number of patients (n=65). Although the ‘best-fit’ (highest combined sensitivity and specificity) cut-off value for SPI to predict moderate-to-severe pain in the current, prospective study was nearly identical to the one reported previously⁹ (29 vs 30), the PPV for SPI >30 calculated under the a priori assumption of this value was only 60%.

This stark discrepancy confirms that data derived from post hoc calculations in small investigations, unfortunately most of the related trials, may be an unreliable reflection of the actual population.

The results of the current investigation need to be seen in context with the type of anaesthesia used by us (sevoflurane-fentanyl). As trials comparing the use of intraoperative SPI-guided analgesia have found different results for total i.v. vs volatile-based anaesthesia,^{6 8} it is likely that the predictive value of SPI for postoperative pain is also influenced by the anaesthetic technique.

In the current study, age had no influence on the prevalence and severity of acute postoperative pain. This is in contrast to previous publications reporting such interactions, with patients of a higher age being more likely to be in pain for longer.11, 12 However, reports of a positive correlation between age and pain are inconsistent, with others finding age to have no influence on postoperative pain.¹³

We also did not find any interaction between age and the accuracy of SPI for the prediction of postoperative pain. Though HR variability, at least one of the parameters of SPI, is likely to be influenced by age;¹⁰ this effect may be of limited clinical importance. As the complete algorithm for the generation of the SPI score is GE-proprietary and as such unpublished, no firm assumption can be made as to why age does not appear to influence SPI.

However, the current study did only include adult patients and can therefore not rule out such an influence when taking children into account. In fact, a recent own investigation of the predictive value of SPI in different aged children concluded that SPI performed differently across the age groups.¹⁴

Though all aforementioned results show significant limitations in the value of SPI for the prediction of postoperative pain, it is worth keeping in mind that we also confirmed the even lower value of MAP and HR for this task, with these parameters being as predictive for postoperative pain as tossing a coin. Similar results have been reported previously.¹⁵

Despite its limitations, it may be the poor performance of the ‘traditional’ parameters such as MAP and HR which create an ongoing significant interest in SPI and similar tools to monitor nociception. Though closed-loop anaesthesia systems using either bispectral index alone or combining it with MAP or HR-based analgesia scores have been found feasible,16, 17, 18 the analgesia component of such systems may require further research.¹⁹ In this context, SPI and similar scores should be trialled to investigate whether these may offer a higher accuracy and reliability than MAP and HR-based scores. Such studies may best include outcomes associated with reduction of related adverse outcomes, such as the transition from acute to chronic pain.

The current results need to be seen in the context of the study’s limitations. We only included adult patients under volatile-based general anaesthesia, with patients on any medication or in intraoperative positions (i.e. step tilt) suspected or known to interact with the assessment of SPI excluded. Hence, we cannot conclude about the ‘performance’ of SPI under conditions of ‘real life’.

Conclusion

Tested a priori in 200 adult patients, a cut-off value of SPI=30 measured at the end of surgery predicted moderate-to-severe postoperative pain with higher accuracy than MAP or HR. However, the PPV and NPV were overall low, and a previously published high PPV could not be confirmed.

Authors’ contributions

Significantly contributed to both study and manuscript: all authors

Study design: TL, JH, MG

Data analysis: TL, MS

Manuscript preparation: all authors

Data collection: RKG, SRT

Declaration of interest

TL and MG and/or their respective institutions have received honoraria and travel funds from GE Healthcare, the manufacturer of SPI. However, neither GE Healthcare nor any other third party had any influence on the conduct, analysis and reporting of this study. All other co-authors declare no conflict of interest.

Funding

Department of Anaesthesia, Royal Perth Hospital, Perth, WA, Australia.

Editorial decision: 31 October 2018

Handling editor: L. Colvin

Appendix A. Supplementary data

The following is the Supplementary data to this article:

Multimedia component 1