Evaluation of noninvasive blood pressure in anesthetized horses: Bias, limits of agreement, and comparative detection of a predetermined mean blood pressure warranting treatment

Abstract

Oscillometric blood pressure monitoring may be a practical tool for short procedures or those performed outside of the operating room. Oscillometric and direct blood pressure values in 30 juvenile and adult horses in a clinical setting using mixed effect models were compared. The limits of agreement and percentage errors were also calculated. We evaluated the sensitivity and false positive rate for the oscillometric method to trigger an intervention for treating blood pressure [direct mean arterial pressure (MAP) < 70 mmHg]. Oscillometric MAP and diastolic arterial pressure (DAP) differed from direct values (P < 0.001); systolic arterial pressure (SAP) did not (P = 0.08). Wide limits of agreement were observed. Percentage errors were smaller for SAP (39%) than for MAP and DAP (48% and 72%). The oscillometric monitor indicated there was a requirement for blood pressure treatment with a true positive rate of 82%, consequently, it failed 18% of the times. The false positive rate (unnecessary treatment) was 55%.

Introduction

Decreases in arterial blood pressure during general anesthesia have been associated with worsening outcomes, including higher risk of postanesthetic myopathy (1) and death in horses (2). Arterial blood pressure during general anesthesia in horses is typically measured directly by inserting a catheter into a peripheral artery. Direct measurements provide accurate, real-time blood pressure values and allow for quick intervention when necessary. Indirect methods such as oscillometric measurements may be desirable under particular situations, including short procedures, or those performed outside of the operating room.

Various oscillometric devices have been tested against direct arterial measurements in standing and anesthetized horses, with variable results in terms of mean differences, bias, and limits of agreement (LOA) between both methods, only involving a few animals (3–5). A recent 2-part review of the methods for noninvasive blood pressure monitoring in various species, including oscillometry in horses, examines current information and provides normal blood pressure values (6,7).

The objective of this study was to assess the performance of a veterinary oscillometric monitor in reference to simultaneous direct arterial blood pressure measurements during anesthesia in horses using many clinical cases. To this end the bias, LOA, and mean difference between both monitors were evaluated in a clinical setting. In addition, the reliability of this indirect method was analyzed for decision-making regarding arterial blood pressure in horses. Specifically, the sensitivity and false positive and negative rates for detecting a mean blood pressure reading of < 70 mmHg were measured with this device, in reference to direct values.

Materials and methods

This study was approved by the Cornell University Veterinary Clinical Studies Committee (protocol #102219-11). Thirty client-owned horses admitted for procedures requiring general anesthesia at the Cornell University College of Veterinary Medicine Equine Hospital were included. Horses of any age, weight, sex, or breed were eligible for this study; horses scheduled for surgical procedures of both dependent limbs while placed in lateral recumbency were excluded.

All horses were fasted from solid food for at least 6 h before administration of anesthesia. After clipping and aseptically preparing the skin, a 14-G, 5.25-inch catheter was placed in a jugular vein. Twenty-five horses received detomidine (Dormosedan; Zoetis, Kalamazoo, Michigan, USA), 5 to 10 μg/kg body weight (BW), IV, before induction of anesthesia. The remaining 5 horses were sedated with xylazine (X-Ject E; Henry Schein Animal Health), 0.5 mg/kg BW, IV before induction of anesthesia. Fourteen horses received morphine (West-Ward, Tinton Falls, New Jersey, USA), 0.05 to 0.1 mg/kg BW, IV, during anesthesia. General anesthesia was induced with ketamine (Ketathesia; Henry Schein Animal Health, Dublin, Ohio, USA), 2.2 mg/kg BW, IV, and midazolam (Midazolam Injection; Alvogen, Pine Brook, New Jersey, USA), 0.1 mg/kg BW, IV. A cuffed orotracheal tube was placed, and anesthesia was maintained with isoflurane in oxygen, and volume-controlled positive pressure ventilation. Monitoring (Cardell Touch; Veterinary Monitor, Midmark Corporation, Ronkonkoma, New York, USA) included continuous electrocardiogram (ECG), inspired and expired concentration of anesthetic agent, pressure of carbon dioxide, pulse oximetry, and arterial blood pressures (direct and oscillometric). Periodical arterial blood gas analysis was also performed (i-STAT1 Analyzer; Abbott Point of Care, San Diego, California, USA). All horses received an infusion of a balanced crystalloid solution at approximately 10 mL/kg BW per hour (Veterinary Plasma-Lyte A; Zoetis, Kalamazoo, Michigan, USA) throughout the procedure. Dobutamine (Hospira, Lake Forest, Illinois, USA), up to 2 μg/kg BW per minute, or detomidine (Dormosedan; Zoetis), 0.1 μg/kg BW per minute, was administered at the discretion of the attending anesthesiologist.

Blood pressure monitoring and data collection

A 20-G 1.25-inch or 22-G 1-inch catheter (Covidien Monoject; Henry Schein, Mansfield, Massachusetts, USA) was placed on a facial or transverse facial artery for horses undergoing surgeries of the abdomen or hind limbs, or on a metatarsal artery for horses undergoing front-limb or airway surgeries. The catheter was connected to a blood-pressure transducer (Deltran IV disposable pressure transducer; Utah Medical Products, Midvale, Utah, USA) via a low-volume, non-compliant tube, filled with normal saline containing heparin (Heparin Sodium; Hospira). The transducer was placed at the same height as the oscillometric cuff and zeroed to atmospheric pressure.

An oscillometric cuff (Large Animal BP cuff; Sharn Anesthesia, Caledonia, Michigan, USA) was placed either over a meta-carpal or metatarsal artery. Cuff size was determined as suggested by the monitor’s manufacturer, whereby the width of the cuff approximated 40 to 60% of the limb’s circumference. Oscillometric blood pressure was measured automatically every 3 min.

Direct and indirect systolic, mean, and diastolic arterial pressures (SAP, MAP, and DAP) were measured and stored by the multiparameter monitor. Accuracy of the invasive blood pressure was verified using a blood pressure transducer simulator (BP-28 Pressure Transducer Simluator; Fogg System Company, Aurora, Colorado, USA) with values of 0, 10, 50, 100, and 300 mmHg. Function of the oscillometric module was verified with a NIBP simulator (AccuPulse Benchtop NIBP; Clinical Dynamics, Plantsville, Connecticut, USA). Invasive blood pressure values were automatically stored every minute, but only pairs of simultaneously recorded values (every 3 min) were used for analysis.

Statistical analysis

Descriptive statistics of the demographical characteristics of the horses are presented [median (minimum – maximum)]. A failure of the oscillometric monitor was recorded when no value was determined with this method during an automatic cycle. The distribution of the residuals was observed, and values for SAP, MAP, and DAP were compared between methods with mixed effect models, using horse as the random effect and method as the fixed effect (8). The mean difference of all pairs was also calculated. These data are summarized as mean ± standard deviation (SD), with significance set at P < 0.05. Bland-Altman plots were used to calculate the bias and LOA for SAP, MAP and DAP between monitors by plotting the difference (invasive – oscillometric) over the average of each pair (9). The percentage error was calculated as 2 × SD of bias/mean (10).

Using the simultaneous direct and oscillometric MAP values to evaluate the ability of the oscillometric monitor to trigger an intervention for treating low blood pressure, a receiver operating characteristics (ROC) curve was created (11). For this, a MAP < 70 mmHg measured with the direct method was selected as a threshold; values < 70 mmHg would trigger treatment to raise the blood pressure. The ROC curve is a plot of the sensitivity (true positive rate) against 1-specifity (false positive rate) of the oscillometric method for diagnosing a blood pressure below the intervention threshold. The area under the ROC curve was also calculated.

Statistical analyses were performed with JMP Pro 15 (SAS Institute, Cary, North Carolina, USA) and RStudio (Rstudio, Integrated Development for R, Rstudio, Boston, Massachusetts, USA).

Results

Data were obtained from 17 male (2 intact) and 13 female horses with an overall age range of 0.3 to 25 y and average weight 470 kg (range: 170 to 1001 kg). Breeds included 7 Thoroughbred, 17 Standardbred, 1 Hanoverian, 1 Friesian, 1 Oldenburg, 1 Percheron, 1 Warmblood, and 1 Quarter Horse. Twenty-eight horses were placed in dorsal recumbency, and 2 horses were each placed in lateral recumbency. Mean duration of anesthesia was 2 h 27 min (range: 1 h 25 min to 5 h 17 min).

A total of 963 paired values were recorded. Failures during oscillometric measurements occurred in 17/30 horses. The overall failure rate considering all 30 horses was 4% (range: 0 to 73%). In those 17 animals in which failures occurred, the failure rate was 14% (range: 3 to 73%). After exclusion from failures, a total of 837 pairs were obtained (804 from horses in dorsal recumbency and 33 from horses in lateral recumbency).

Significant differences between methods were discovered with mixed effect models for MAP and DAP (both P < 0.001); however, there were no differences for SAP (P = 0.08). Values are summarized in Table 1.

Table 1.

Mean systolic, mean, and diastolic blood pressures measured directly and with an oscillometric monitor in 30 anesthetized horses.

	Direct blood pressure	Oscillometric blood pressure	Mean difference	P-value
SAP (mmHg)	103 (12)	102 (22)	1 (20)	0.08
MAP (mmHg)	75 (11)	67 (40)	8 (40)	< 0.0001
DAP (mmHg)	61 (12)	47 (17)	14 (17)	< 0.0001

The mean difference between monitors is shown. The significance of differences between monitoring methods was assessed with mixed effect models.

Bland-Altman plots with the bias and LOAs are shown in Figure 1. The percentage errors were 39%, 48% and 72% for SAP, MAP, and DAP, respectively.

Figure 1.

Bland-Altman plots of the difference between direct and oscillometric pairs of systolic (top), mean (middle), and diastolic (bottom) (SAP, MAP, DAP) blood pressure measurements over the average in 30 anesthetized juvenile and adult horses. The bias (solid line), and the upper and lower limits of agreement (LOA) (dotted lines) are indicated for each plot.

The ROC curve is shown in Figure 2. When using a measured direct MAP < 70 mmHg as a threshold for supporting blood pressure, the oscillometric monitor had a true positive rate of 82% (that is, it failed to trigger intervention in 18% of cases requiring intervention) and a false positive rate (triggered intervention when this would not be needed) of 55%. The combination of false positives and false negatives expected when using an oscillometric MAP of 70 and 80 mmHg as triggers for intervention are identified in the graph. The area under the curve was 0.72.

Figure 2.

Receiver operating characteristic (ROC) curve of the ability of oscillometric mean arterial pressure (MAP) to detect hypotension in anesthetized juvenile and adult horses. The combination of true positive rate and false positive rate for detecting hypotension when using oscillometric values of 70 and 80 mmHg are indicated. The area under the curve was 0.72.

Discussion

The current study evaluated the performance of oscillometric blood pressure with the Cardell monitor compared to direct measurements in anesthetized horses. Oscillometric blood pressure monitoring is often used as an alternative to the more invasive and technically more difficult direct method, whereby catheterization of a peripheral artery is required. However, acceptable agreement is required between both methods before an alternate method can replace the reference one. The level of disagreement that might be accepted between 2 methods is a clinical question rather than a statistical one; to that end, we used different statistical techniques to compare both monitoring modalities.

A high failure rate occurred with the oscillometric method, whereby the device was unable to produce values at the expected time during automatic cycling. Failures occurred in 17/30 horses, with a high variability in the rate of failure, which ranged between 3% and 73% in those animals. Hatz et al (3) reported an overall failure rate of 21.7% with a different oscillometric device in anesthetized horses. Although our data do not allow us to discern the cause of this problem in some animals, it appears that this issue is not exclusive to this monitor but to the oscillometric methodology in general.

When monitors were compared using a mixed effect model, as recently suggested in an opinion article (8), significant differences were recorded between methods for MAP and DAP, but not SAP. The difference between methods for the latter was only 1 mmHg, albeit, with a large standard deviation. The national standard for measurement of blood pressure with automated sphygmomanometers in humans (12) states that the mean difference between pairs should be no more than 5 mmHg with an SD of no more than 8 mmHg. Those guidelines were not fulfilled by SAP despite the lack of statistical difference between methods. The differences for MAP and DAP were increasingly larger; interestingly, the largest SD was observed for MAP.

As with the trend observed in mixed effect models, bias, LOA, and PE calculated from Bland-Altman plots were smaller for SAP than for MAP. This observation was unexpected as, generally, the MAP is considered the most reliable value as it is calculated from the largest oscillations (6,7,13,14). The increased variability in MAP poses a clinical problem, as this parameter is ordinarily used to intervene when hypotension ensues. Moreover, in horses, associations between arterial hypotension and negative outcomes are typically documented in reference to MAP (1,2,15). However, despite this apparent superior performance of the oscillometric SAP, it should be noted that the LOA and PE for that variable were likely too wide to be considered a reliable surrogate for direct blood pressure measurements; therfore, we are reluctant to recommend the use of SAP over MAP.

The usefulness of the oscillometric method to trigger interventions using an ROC curve was also evaluated. For this purpose, a threshold of 70 mmHg was selected for MAP measured with the direct method, whereby values below that threshold would trigger a response. This device had a true positive rate of 82%, that is, it correctly triggered an intervention 82% of the time. Consequently, this monitor would fail to trigger an intervention 18% of the time, in the case that a direct MAP < 70 mmHg is considered the intervention point. The false positive rate was higher, at 55%, meaning that it displayed an oscillometric MAP < 70 mmHg when the direct measurement was ≥ 70 mmHg. Such error would result in the unnecessary treatment of hypotension. As observed from the ROC curve, the true positive rate can be improved by using a higher threshold for triggering intervention with the oscillometric monitor. As an example, if blood pressure support was initiated when the oscillometric MAP was < 80 mmHg, then the sensitivity of this monitor for this purpose increases from 82 to 97%. However, as expected, this improvement in the sensitivity occurs with an increase in the false positive rate. The ROC curve provides a graphical representation of how the false positive rate increases as the sensitivity is augmented. The area under the ROC curve is a combined evaluation of sensitivity and 1-specificity, and represents the overall performance of the diagnostic test, in this case, oscillometric blood pressure monitoring; the closer to 1 this area is, the better the performance of the test. The area under the curve in our study was 0.72, a value which is intermediate between the best possible performance (an area of 1), and a diagnostic performance equal to chance (area of 0.5). Traditionally, when evaluating a device or technique as a diagnostic tool (in this case, to diagnose the need for blood pressure support), areas under the ROC curve between 0.7 to 0.8 are considered acceptable, 0.8 to 0.9 is considered excellent, and > 0.9 is considered outstanding (11).

For this study, the cuff of the oscillometric monitor and the transducer for direct measurements were placed at the same height, which in some cases, resulted in locations higher than the estimated location of the right atrium. Such location will invariably result in blood pressure values that underestimate the true measurement of blood pressure. It should be noted, therefore, that the threshold for intervention of 70 mmHg that we selected may not truly be an accurate measurement in some animals. However, this artifact is introduced to both monitors by an equal magnitude such that no one monitor is biased. As stated, the main objective of the study was to assess the performance of the device, not the precise blood pressure values in these horses. An alternative to this method is to measure the vertical distance between a correctly positioned and zeroed blood pressure transducer and the oscillometric cuff and correct the displayed value by converting the distance into mmHg (4,5). With this approach, the transducer can be zeroed at the expected height of the right atrium, and a correction factor is applied to adjust for the height of the oscillometric cuff. Evaluation of the performance of the oscillometric method only applied to the limbs. Other locations might be used, such as the tail, which, for horses in lateral recumbency, might be situated closer to the expected height of the heart. Such instrumentation has been used with success in horses (16). For this study, arterial catheters were placed at different sites depending on the procedure being conducted. A study in horses has shown that there might be differences in blood pressure measured from different arteries (17). However, placement at different sites is representative of daily clinical practice, therefore, this study provides useful clinical information.

There are some limitations to this study. Data were collected from clinical cases, and there were no blood pressure manipulations done to obtain low values, which may be possible in experimental animals. Hypotension was promptly treated; therefore, extreme values did not occur, and the number of hypotensive measurements was minimized. Additionally, blood pressure was measured at different sites. Comparisons between sites was not performed due to the unbalanced numbers between potential groups. Despite these limitations, these data reflect real-case scenarios in clinical practice.

Oscillometric blood pressure measured at a limb of anesthetized horses showed wide LOA when compared with direct values and was statistically different from the gold standard. The oscillometric monitor could correctly trigger treatment to increase blood pressure detected hypotension, defined as a MAP < 70 mmHg (with our transducer placement) 82% of the time. CVJ