Diagnostic decision rule for support in clinical assessment of the need for surgical intervention in horses with acute abdominal pain

Abstract

A prospective survey of horses with colic referred to a university hospital was undertaken to elaborate on a simple clinical decision support system capable of predicting whether or not horses require surgical intervention. Cases were classified as requiring surgical intervention or not on the basis of intraoperative findings or necropsy reports. Logistic regression analysis was applied to identify predictors with the strongest association with treatment needed. The classification and regression tree (CART) methodology was used to combine the variables in a simple classification system. The performance of the elaborated algorithms, as diagnostic instruments, was recorded as test sensitivity and specificity. The CART method generated 5 different classification trees with a similar basic structure consisting of: degree of pain, peritoneal fluid colour, and rectal temperature. The tree, constructed at a prevalence of 15% surgical cases, appeared to be the best proposal made by CART. In this classification tree, further discrimination of cases was obtained by including the findings of rectal examination and packed cell volume. When regarded as a test system, the sensitivity and specificity was 52% and 95%, respectively, corresponding to positive and negative predictive values of 68% and 91%. The variables examined in the present study did not provide a safe clinical decision rule. The classification tree constructed at 15% surgical cases was considered feasible, the proportion of horses incorrectly predicted to be without need of immediate surgery (false negatives) was small, whereas the proportion of horses incorrectly predicted to be in need of immediate surgery (false positives) was large. Some of the false positive horses were amenable to surgical treatment, although these cases did not conform to the strict definition of a surgical case. A less rigorous definition of a surgical case than that used in the present study would lower the percentage of false positives.

Introduction

Choosing the correct treatment within a very short space of time is one of the dilemmas facing clinicians working in equine colic referral centers. The number of horses with colic that have severe gastrointestinal lesions, which rapidly lead to circulatory collapse and finally death, is generally high in equine hospitals (1,2,3,4). An accurate diagnosis, based on the clinical examination carried out at admission, is often difficult to obtain. Blikslager and Roberts (3) examined the accuracy of clinicians in predicting the site and type of lesion, and found a correct classification of 41% in 96 horses with colic. To facilitate early treatment, diagnosis is best restricted to simple discrimination between cases needing surgical correction and cases treatable with medicine alone. Both experimental (5,6,7,8) and clinical research (9,10) document the usefulness of individual clinical and laboratory variables in distinguishing medical and surgical cases of colic. To assist clinical assessment, Ducharme et al (11) developed a decision algorithm based on data collected from 219 horses with colic. Abdominal distension was reported to be the most discriminatory variable, followed by rectal examination, and peritoneal fluid colour. In a multicenter study (12) a logistic regression model was built on data from 640 cases of colic from 10 equine referral centers in the United States. Rectal findings, signs of abdominal pain, peripheral pulse strength, and abdominal sounds were the variables resulting from the multivariate analysis. Freden et al (13) examined the reliability of peritoneal fluid variable in 218 horses with colic as possible predictors of treatment and lesion type. When the variables were combined in a logistic regression analysis, only specific gravity and peritoneal fluid colour were significant. It was concluded that peritoneal fluid analysis alone does not provide satisfactory guidance in determining the appropriate management of horses with colic. The objective of the present study was to develop a predictive model for use in clinical identification of horses with colic requiring surgical intervention.

Materials and methods

Data

Demographic, clinical, and clinical-chemical data were collected prospectively for all horses with colic admitted to the Royal Veterinary and Agricultural University of Copenhagen, Denmark between August 1994 and December 1997. All samples and clinical measurements relating to an individual horse were taken simultaneously and only once, according to standard procedure. Samples were stored and processed as prescribed by the central laboratory at the university. If fecal material was present in peritoneal fluid samples, no further analyses were performed.

Horses were classified as surgical cases only if surgical intervention was invariably necessary. Surgical intervention was considered invariably necessary if a strangulating lesion, intestinal infarction, or small intestinal impaction was positively diagnosed either intraoperatively or at necropsy. Horses requiring explorative laparotomies in the later course of an episode of colic; horses treated successfully with medicine alone; and horses where necropsy did not reveal any strangulation, infarction, or small intestinal impaction, were classified as not requiring surgical correction. Cases were excluded from analysis where post-mortem records were considered to be insufficient for reliable classification. Horses that had gastric or intestinal ruptures, or had lesions considered to be primarily extra-enteral in nature (for example, through uterine torsion), were also excluded. All of the horses that died were subjected to routine autopsies by the Department of Veterinary Pathology at the university and a post-mortem record was prepared for each case.

Statistical analysis

The simple association between type of treatment needed as the response variable and the individual variables was examined using graphs prior to any analysis. If no linear trend was apparent, continuous variables were categorized. Bivariate logistic regression (Statistical Analysis System, version 6.12; SAS Institute Inc., SAS Campus Drive, Cary, North Carolina, USA) was used as a screening method to identify the variables related to the treatment needed and to obtain odds ratio estimates of this association. Variables with P < 0.15 were considered potential candidates for the following multivariate analysis. Due to a substantial proportion of missing values only variables that had more than 200 observations and were important in the initial screening (P < 0.15), were included in the multivariate analysis.

The classification and regression tree method (CART) (CART, version 1.1; California Statistical Software Inc., Lafayette, California, USA) elaborated by Breiman et al (14) was developed to combine multiple variables in a classification tree. Briefly, CART is a non-parametric technique for dividing a population or sample into subgroups. For any given event (for example, surgical case status), the aim is to distinguish a pair of groups in which the number of individuals possessing the event is maximized and minimized, respectively. The selection of the explanatory variable that yields the best split (the 1st branch point) is made by comparing the capacity of the different variables in order to identify the most homogeneous subgroups. Likewise, the actual position of the split in the variable range is determined by comparisons of successive splits in the scale. The splitting criterion used in the present study is the so-called Gini index (15). Subsequent division of the subgroups created by the 1st split occurs in a similar fashion and proceeds until a stopping criterion is met. In the present study, no further partitioning was attempted when a node consisted of 5 cases or less. If missing values are present in the variable used in a split, CART identifies a surrogate variable, which yields a comparable splitting and allows the construction of the tree to continue. A comprehensive account of the methodology is provided by Feinstein (15).

The 95% confidence interval for the prevalence of surgical cases in the present study was 14 to 20%. In CART, the prior probability of an event (for example, the prevalence of disease) can be changed using an optional setting of the procedure. This option was used to examine whether CART proposed different solutions when the a priori probability varied from 14 to 20%. Honest estimates of the performance of the resulting trees were made by 10-fold cross validation (14). The performance of each tree as a diagnostic instrument was recorded as test sensitivity and specificity.

Results

A total of 528 horses with colic were admitted to the Large Animal Hospital during the study period. Thirty-eight horses were excluded from the analysis: 17 of these had an intestinal rupture, 16 had miscellaneous extra-enteral conditions, and 5 could not be classified because they had insufficient post-mortem records. Of the 490 horses with colic remaining, 84 were classified as requiring surgical intervention, thus the prevalence of surgical cases was 17% (the approximate 95% confidence interval was 14 to 20%). Of these 84 horses, 4 suffered from ileal impactions; 2 had intestinal infarctions; and 78 suffered from different types of strangulating lesions. Small intestinal involvement was found in 56 surgical cases, 26 horses had large intestinal lesions, and in 2 horses the lesion site was the small colon.

A large number of the variables examined in the present study were considered significantly associated (P <0.15) with treatment needed in the bivariate analysis. In Tables I and II the P-values and strengths of these bivariate relations are shown as odds ratios (OR). For continuous variables the OR reflects the increase or decrease in the risk of surgery being necessary following 1 unit change in the variable scale. For categorical variables the OR reflects the change in the risk of surgery being necessary as compared with the reference category; OR = 1 indicates no association, OR > 1 indicates a direct (predisposing) association, whereas OR < 1 indicates an inverse (protective) association.

Table I.

Table II.

The following demographic variables showed no strong association (P > 0.15) with treatment required: admission time (day, evening, night); admission date (month, year); gender; breed; and age of the horse. Likewise, no strong association was found for the following blood (plasma) variables: hemolysis, potassium, calcium, magnesium, phosphate, alkaline phosphatase, and gamma glutamyltransferase. Further, cecal decompression, peritoneal fluid turbidity, peritoneal fluid calcium, and the blood-peritoneal fluid ratios of calcium and chloride showed no strong relation to the treatment needed.

Of the variables where P < 0.15 in the bivariate association, only 14 had more than 200 observations (Tables I and II) and therefore entered the multivariate analysis. These were: pre-hospital analgesic medication, degree of pain, mucous membrane colour, abdominal sounds, gastric reflux, rectal examination, peritoneal fluid hemolysis, heart rate, capillary refill time, rectal temperature, packed cell volume, plasma protein, standard base excess, and peritoneal fluid protein. The CART method generated 5 distinct classification trees for the 7 different prevalences in the confidence interval (14 to 20%). The trees constructed for 15%, 16%, and 17% of surgical cases are shown in Figures 1, 2, and 3. The simplest tree developed at a prevalence of 17% consisted of 3 splits at the following variables: degree of pain, peritoneal fluid hemolysis, and rectal temperature. This basic structure could be identified in the 6 trees developed in 14 to 19% of surgical cases. In fact, the tree created for a prevalence of 17% was identical to the trees constructed at 18% and 19%. The tree, at a prevalence of 14% was identical to the tree developed at 15%, except in the respect that further partitioning was discontinued if no peritoneal fluid hemolysis was evident. At 20% the tree simply traced degree of pain and peritoneal fluid hemolysis.

Figure 1. Classification trees for the assessment of the treatment type needed as proposed by CART at prevalences of surgical cases of 15%. Ellipses represent splitting nodes and the rectangles represent the terminal nodes. The numbers shown under each node designate cases reaching this node. The first number is that of cases requiring surgery and the second number records cases not needing surgical treatment. The percentage enclosed in ellipses and rectangles is the probability that surgery is needed, in the node. Terminal nodes in bold are surgical end categories. For prevalences of surgical cases other than 17%, CART operates with a corrective factor and therefore the proportion of surgical horses in each node does not correspond to the relevant probability.

SE — sensitivity of the tree

SP — specificity of the tree

Pf — peritoneal fluid

PCV — packed cell volume

Figure 2. Classification trees for the assessment of the treatment type needed as proposed by CART at prevalences of surgical cases of 16%. Ellipses represent splitting nodes and the rectangles represent the terminal nodes. The numbers shown under each node designate cases reaching this node. The first number is that of cases requiring surgery and the second number records cases not needing surgical treatment. The percentage enclosed in ellipses and rectangles is the probability that surgery is needed, in the node. Terminal nodes in bold are surgical end categories. For prevalences of surgical cases other than 17%, CART operates with a corrective factor and therefore the proportion of surgical horses in each node does not correspond to the relevant probability.

SE — sensitivity of the tree

SP — specificity of the tree

Pf — peritoneal fluid

PCV — packed cell volume

Figure 3. Classification trees for assessment of treatment type needed as proposed by CART at prevalences of surgical cases of 17%. Ellipses represent splitting nodes and the rectangles represent the terminal nodes. The numbers shown under each node designate cases reaching this node. The first number is that of cases requiring surgery and the second number records cases not needing surgical treatment. The percentage enclosed in ellipses and rectangles is the probability that surgery is needed, in the node. Terminal nodes in bold are surgical end categories.

SE — sensitivity of the tree

SP — specificity of the tree

Pf — peritoneal fluid

The specificity for trees constructed at 14 to 20% was almost identical (range: 92 to 96% after cross-validation). The sensitivity of the tree constructed at a prevalence of 14% was especially low (44%), whereas the 6 other trees had sensitivities in the range of 49 to 57% after cross-validation. The sensitivities and specificities for the trees developed at a prevalence of 15 to 17% are shown in Figures 1, 2, and 3. Final diagnoses of the false positive and false negative classifications of horses in the tree developed at prevalence 15% are presented in Table III.

Table III.

Discussion

Multivariate analysis has been used to assess surgical need in 2 previous studies (11,12). In these studies “surgical cases” were defined as those involving horses with intestinal lesions (identified intraoperatively or at necropsy) that required surgical correction or could benefit from surgery. In the present study a more restrictive definition of a surgical case was chosen, since the aim was to identify those horses invariably requiring surgery. Without surgery, a strangulating obstruction or non-strangulating infarction inevitably leads to tissue necrosis and finally to death. Hansson et al (16) reported that small intestinal impaction could very occasionally be treated medically with success. However this option was only suggested where there was no surgical alternative. In horses with ileal impaction, the survival rate rapidly decreases in inverse proportion to the duration of clinical signs (17). These 3 diagnostic categories were therefore used as classification criteria for the surgical cases in the present study.

For all the classification trees generated in the present study by CART, the first branchings were created by distinguishing horses with no, mild, or moderate pain from those that were lethargic or in severe pain. Research and clinical experience has shown that intensity of pain is one of the most important variables in discriminating surgical and medical cases of colic in horses (18,19,20,21,22). The importance of this variable, in combination with other clinical information, is confirmed by Reeves et al (12).

The identification of rectal temperature as a factor significantly associated with the type of treatment needed, has not been previously reported. In horses with colic, changes in body temperature can be attributed to increased activity caused by pain, local or systemic inflammatory response, or insufficient cardiovascular function. The bivariate analysis suggests that there is an inverse association between temperature and the need for surgical intervention (Table II). Thus an increase in rectal temperature decreases the risk that surgery is needed. However, CART only uses this information in connection with lethargic horses or horses in severe pain (Figures 1, 2, and 3). This means that interaction between the variables, degree of pain and rectal temperature, is present. In the clinical setting the authors have previously, as a rule of thumb, regarded temperatures of 39°C or higher to be symptoms of enteritis or peritonitis and therefore inconsistent with a surgical lesion. A review of earlier reports on peritonitis found that pyrexia was present in 57% of the horses (23). Cohen and Woods (24) reported that 20 out of 122 horses with acute diarrhea had rectal temperatures above 38.6°C. Thoefner et al (4) have recently postulated a more complex association between outcome (survival or death) and temperature than had previously been suggested.

The capacity of peritoneal fluid color to distinguish between surgical and medical cases of colic has been shown in a number of studies (13,20,25). In the present study, only in horses with moderate or minor pain, was further partitioning by peritoneal fluid hemolysis obtained. The present finding that peritoneal fluid color was indicative, primarily in horses exhibiting milder symptoms, was in agreement with the classification algorithm developed by Ducharme et al (11). In horses without moderate or severe abdominal distension, where rectal palpation revealed no intestinal distension, peritoneal fluid colour was of further help in the separation of surgical and medical cases. The clinical implications of this finding are significant, since abdominocentesis can be quite troublesome in horses with colic that are responding poorly to analgesics. As in the present study, the study conducted by Ducharme et al (11) showed that evidence of peritoneal fluid hemolysis increased the likelihood that surgery would be necessary. A few cases were misclassified by peritoneal fluid hemolysis in both studies. However, peritoneal fluid cannot always be obtained through abdominocentesis. In a study undertaken by Siex and Wilson (26), fluid was unobtainable in 13% of cases, despite several attempts. The advantage of CART as an analytical method becomes evident here, since in situations like this it generates a surrogate split. When missing values exist in peritoneal fluid hemolysis, CART suggests rectal temperature as a surrogate variable (Figures 1, 2, and 3). Although identified as a useful surrogate variable, rectal temperature was a much weaker discriminator (data not shown) than peritoneal fluid hemolysis. Thus, the high number of missing peritoneal fluid observations is likely to have a negative influence on model performance in the present study. Likewise, other variables in a strong association with the treatment needed, but with too few observations to enter the multivariate analysis, might possess additional discriminatory power.

Rectal examination is considered to be the procedure most helpful in diagnosing horses with colic. Specific diagnoses can be made, but more often the recognition of intestinal distension or dislocation is a clear indication that a case belongs to a certain diagnostic category (27,28,29). In the present study, rectal findings were coded as either normal, involving large intestinal impaction or coprostasis, small intestinal distension, or some other abnormal finding. Obviously, this simplification results in a loss of information. However, rectal palpation still provides further partitioning in the trees constructed at prevalence of 14 to 16%. The ability of rectal palpation to discriminate between surgical and medical cases of colic in a multivariate analysis was also shown by Ducharme et al (11) and Reeves et al (12). Variables reflecting the condition of the cardiovascular system are associated with both the prognosis and treatment needed (9,20,30). In a multivariate model (12) a weak peripheral pulse increased the risk that surgery was necessary. In the present study, packed cell volume enabled further splitting of the cases in some of the distal branches of the trees. This was seen in the classification trees developed at a prevalence of surgical cases of 14 to 16%. For horses following certain classification paths, the probability of surgical treatment being necessary increased where high packed cell volumes were present (Figures 1 and 2).

The CART method proposed 5 distinct classification trees for the 7 different prevalences at 14 to 20%. Apart from the very simple tree developed at a 20% prevalence of surgical cases, the basic structure of the trees is similar. When applied as test systems, the differences in sensitivity and specificity in these trees are small, with the exception of the tree constructed at 14%, which had lower sensitivity. Clinical experience suggested to the authors that the basic structure itself would be insufficient for the purpose of classification by the type of treatment needed. Thus, the trees constructed at 17 to 20% were considered to be too simple. The authors also fully concur with the existing literature on the diagnostic benefits of rectal examination. The classification tree developed at a prevalence of 15% surgical cases (Figure 1) was therefore selected as the most reliable proposal generated by CART. For this tree the sensitivity and specificity found by cross-validation were 52% and 95%, respectively. Using these figures in the calculations described by Henken et al (31), the positive and negative predictive values were 68% and 91%. This means that 32% of horses classified as surgical cases did not in fact require surgical intervention (false positives) and that 9% of horses classified as non-surgical cases actually needed surgical correction (false negatives). Although horses in the false positive category are wrongly classified as surgical emergencies, some of them remain treatable with surgical intervention. In Table III, 7 of the 18 horses figuring as false positives (including 2 horses with renosplenic entrapment and 5 with large intestinal impaction) would respond to surgical treatment. Thus, a less rigorous definition of a surgical case than that used in the present study is likely to lower the percentage of false positives. In the study by Ducharme et al (11) the prevalence of surgical cases was 79%. The sensitivity and specificity of the decision algorithm were 99% and 55%, respectively. Equally, the positive and negative predictive values were reported as 90% and 99%, respectively. Differences in classification criteria for surgical cases and prevalence make genuine comparisons problematic, but the sensitivity of the selected tree in the present study was nevertheless disappointing. From Table III, it can be seen that some horses with small intestinal lesions were incorrectly classified as false negatives. An increase in test performance might be obtained if ultrasonographic evaluation of the abdomen was used together with the variables selected by CART in the present study. In the diagnosis of small intestinal strangulation, Klohnen et al (32) reported a sensitivity and specificity of abdominal ultrasonography of 100%, respectively, and a sensitivity and specificity of rectal palpation of 50% and 98%, respectively. Although CART, in the present study, proposes surrogate variables where rectal examination is impossible owing to the size of the horse, the authors suggest abdominal ultrasonographic evaluation as an alternative, since some of the information obtained by both examinations is similar. The benefits of using abdominal ultrasonography in multivariate assessment of required treatment needs further investigation.

Due to the obvious negative consequences of misdiagnosing a horse requiring surgical intervention, the predictive value of the negative test should be high. Although the positive predictive value of the classification tree constructed at 15% is relatively low, the authors believe that the CART methodology could aid in the decision-making process of selecting surgical treatment in horses with acute abdominal pain. However, the test performance needs to be increased and the clinical usefulness of the classification tree developed in the present study needs to be evaluated in a separate validation study. Specifically, a study that objectively compares the accuracy of the predictive model presented in this paper with the abilities of clinicians in predicting the need for surgery in horses with colic, is needed.