The AFC Score: Validation of a 4-Item Predicting Score of Postoperative Mortality After Colorectal Resection for Cancer or Diverticulitis

Abstract

Objective:

The aim of the present prospective study was to validate externally a 4-item predictive score of mortality after colorectal surgery (the AFC score) by testing its generalizability on a new population.

Summary Background Data:

We have recently reported, in a French prospective multicenter study, that age older than 70 years, neurologic comorbidity, underweight (body weight loss >10% in <6 months), and emergency surgery significantly increased postoperative mortality after resection for cancer or diverticulitis.

Patients and Methods:

From June to September 2004, 1049 consecutive patients (548 men and 499 women) with a mean age of 67 ± 14 years, undergoing open or laparoscopic colorectal resection, were prospectively included. The AFC score was validated in this population. We assessed also the predictive value of other scores, such as the “Glasgow” score and the ASA score. To express and compare the predictive value of the different scores, a receiver operating characteristic curve was calculated.

Results:

Postoperative mortality rate was 4.6%. Variables already identified as predictors of mortality and used in the AFC score were also found to be associated with a high odds ratio in this study: emergency surgery, body weight loss >10%, neurologic comorbidity, and age older than 70 years in a multivariate logistic model. The validity of the AFC score in this population was found very high based both on the Hosmer-Lemeshow goodness of fit test (P = 0.37) and on the area under the ROC curve (0.89). We also found that discriminatory capacity was higher than other currently used risk scoring systems such as the Glasgow or ASA score.

Conclusion:

The present prospective study validated the AFC score as a pertinent predictive score of postoperative mortality after colorectal surgery. Because it is based on only 4 risk factors, the AFC score can be used in daily practice.

The present prospective study validated the AFC score as a pertinent predictive score of postoperative mortality after colorectal surgery. Because it is based on only 4 risk factors (emergency surgery, body weight loss >10% in less than 6 months, neurologic comorbidity, and age >70 years), the AFC score can be used in daily practice.

The outcome of surgical treatment depends not solely on the ability of the surgeon but also on the patient's physiological status, the disease that requires surgery, the nature of the operation, and the quality of perioperative care.¹ Predicting the risk of death following surgery with a validated scoring system remains an important part of a surgical audit. It enables clinical decisions to be made on individual patients based on calculations from the preoperative and intraoperative risk factors² so the patient or his family can be well informed of the approximate risk of mortality before surgery. Among the several scoring systems that have been developed over recent years, the “Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity” (POSSUM) scoring system remains the most appropriate for surgical specialties, including colorectal surgery.³ However, POSSUM and the Portsmouth revision (P-POSSUM) have been reported to overpredict both mortality and morbidity rates, particularly in young patients and elective colorectal procedures.^4,5 Recently, Tekkis et al have developed a method of measuring risk-adjusted outcomes in patients undergoing colorectal surgery, using a modification of the POSSUM model specific to colorectal disease (CR-POSSUM).⁶ In their experience, the CR-POSSUM model offered the best overall accuracy compared with POSSUM and P-POSSUM, respectively. However, Senagore et al have reported again an overprediction of mortality for colon cancer resection with all these 3 POSSUM variants (ie, POSSUM, P-POSSUM, and CR-POSSUM).⁷

We have recently reported the results of a French prospective multicenter study including more than 1400 patients, which evaluated both postoperative mortality and morbidity after colorectal resection for cancer or diverticulitis.⁸ In this study, multivariate analysis found 4 independent risk factors for mortality: age older than 70 years, neurologic comorbidity, underweight (body weight loss >10% in <6 months), and emergency surgery. The risk of death ranged from 0.5% to 2% when there was no or one risk factor; it was 10% when 2 factors were present, nearly 20% when 3 factors were present, and as high as 50% when all the 4 factors were present. Furthermore, we noted that comparison of expected with observed mortality using POSSUM score, P-POSSUM, and this 4-item score developed in this study, showed that: 1) the POSSUM system overpredicts significantly the mortality for the whole series; 2) the P-POSSUM system had a better predictive value than POSSUM in this study, but it overpredicted the risk of death by 2-fold after elective surgery; and 3) Association Française de Chirurgie (AFC) score was the best scoring system.⁹ Finally, the POSSUM and P-POSSUM scoring systems are based on complex mathematical formulas, while the AFC score is a simple clinical scoring system with only 4 items.

It is well known that the predictive values of many prognosis scores are generally overestimated by their authors, resulting in disappointments due to their more modest usefulness when they are applied in practice by other groups. The most robust way to ascertain the clinical validity of a prognosis score is to evaluate its properties in a completely independent sample.

So the aim of this new French prospective multicenter study was to externally validate the AFC score in a separate population of patients undergoing colorectal resection for cancer or diverticulitis.

PATIENTS AND METHODS

All members including university, general or private hospitals, of the French Association for Surgery (AFC) were encouraged to participate in a new prospective multicenter study exploring both mortality and morbidity, after colorectal surgery during a 4-month period (from June to September 2004). During the study period, we included all the consecutive patients undergoing open or laparoscopic surgery (electively or on emergent basis) for colorectal cancers or diverticular disease performed at the various centers. Exclusion criteria were colectomy for other causes (eg, inflammatory bowel diseases, benign polyps).

The structured data collection sheet included all 28 data points that were significantly associated by univariate analysis with a higher risk of in-hospital mortality in our previous study⁸: age >70 years, body mass index, Glasgow coma scale, score of the American Society of Anesthesiology (ASA), weight loss of more than 10% within the past 6 months, cardiopulmonary comorbidity (including myocardial infarction, atrial fibrillation, chronic obstructive pulmonary disease, smoking history, cardiac medications, and results of both electrocardiogram and chest radiograph), neurologic comorbidity (including cerebrovascular accident, Parkinson disease, dementia with or without residual neurologic deficit, and functional status), hemoglobin serum level, gamma glutamyl transferase serum level, alkaline phosphatase serum level, albumin serum level, emergency surgery for colorectal cancer (ie, colorectal cancer complicated by obstruction, perforated colorectal cancer), T4 colorectal cancer, absence of prophylactic antibiotics, absence of bowel preparation, emergency surgery (colorectal cancer or sigmoid diverticulitis complicated by obstruction, perforation, or peritonitis), laparotomy versus laparoscopy, fecal contamination, peritoneal metastases, left colectomy, colostomy alone, Hartmann procedure, mean operating time, no intestinal anastomoses, and postoperative surgical complications.

The remaining data included the following:

• Hospital features.

• university, general or private, and the number of patients operated per hospital, respectively.

• Patient features.

• gender, obesity (body mass index >30), blood pressure (mm Hg), pulse (beats/min), diabetes mellitus.

• Disease features.

• Diverticular disease.

• previous episode of diverticulitis (number, treatment of each episode), complicated diverticulitis requiring emergent surgery (abscess, fistula, obstruction, hemorrhage, and perforation).

• Colorectal cancer.

• location, neoadjuvant chemotherapy and/or radiotherapy, complications (including obstruction, tumoral or diastatic perforation, hemorrhage), tumor staging according to TNM classification.

• Surgical procedure.

• intestinal anastomosis (manual vs. stapled, location of the anastomosis: ileocolic, ileorectal, colocolic, colorectal, ileo, or coloanal), protective stoma, duration of operation, amount of homologous blood transfused, blood loss, and abdominal drainage.

• Both postoperative mortality and morbidity were defined as in-hospital death and specific complications, respectively.

Statistical Analyses

Descriptive analysis were made according to survival status and univariate comparisons with t test or χ² tests to check that the variables selected on the basis of their statistical significance in the previous study were also significant in the present validation sample.⁸ In addition, we also checked that the 4 variables constituting the AFC score were still found to be significant in a multiple logistic regression model applied to the present validation sample. No sample size calculation was performed.

Then, the discriminating power of the AFC score simply based on the number of factors (of these 4 variables) present in the patient was validated. For this purpose, mortality rates corresponding to each value of the AFC score and their corresponding odds ratio were calculated.

The discriminatory capacity of the model based on AFC score, which refers to its ability to assign higher probabilities of death to patients who actually die than to those patients who live was estimated by the area under the receiver-operator characteristic (ROC) curve or c-index.¹⁰ Values ranging from 0.7 to 0.8 represent reasonable discrimination, and values exceeding 0.8 represent good discrimination. We also calculated the area under the ROC for the ASA or Glasgow scores, which are other simple scores that can be applied to these patients.

Assessment of the accuracy of the prediction of mortality based on a logistic model using the AFC score was presented on the basis of the comparison between the observed and predicted mortality rates for each value of the AFC score. Statistical validation of the goodness-of-fit of this model based on AFC score was made using the Hosmer-Lemeshow test.^11–13 In addition, we used the statistics proposed by Altman and Royston specifically for quantifying the performance of a score on the basis of an external validation.¹⁴ These authors proposed consideration of a simple separation parameter, which is the difference PSEP = (p_worst − p_best) where p_worst is the mortality rate of patients in the group with the worst prognosis and p_best is the mortality rate of patients in the group with the best prognosis. The comparison of PSEP value calculated in the sample used for building the score and the value calculated in the validation sample allow us to estimate whether the discriminatory capacity of the score was overoptimistically estimated or not.

For all tests, statistical significance was stated as P < 0.05.

RESULTS

A total of 1049 patients operated on in 41 participating centers were included for the analysis of both postoperative mortality and morbidity. The mean inclusion number per center was 25.6 ± 22.2 patients within 4 months (range, 3–108); 788 patients had colorectal cancer (75%) and 261 had diverticular disease (25%). Elective surgery was performed in 870 patients (83%) and emergency surgery in 179 (17%). The rate of patients older than 70 years was 45% (465 of 1049). Surgical procedures included right colectomy (n = 221), left colectomy (n = 481), total colectomy (n = 49), proctectomy with total mesorectal excision (n = 180), abdominoperineal resection (n = 46), Hartmann procedure (n = 63), and colostomy alone (n = 32).

Postoperative Mortality

Forty-eight patients died during their hospitalization after a mean delay of 17 ± 19 days (range, 1–72 days) after operation, given an in-hospital postoperative mortality rate of 4.6%. Univariate analysis showed that the variables selected in the present study because they were significant in our previous study were also significantly associated with a higher risk of mortality in the present validation sample (Table 1). Hospital features were not significantly associated with increased mortality rate.

TABLE 1. Univariate Analysis of Factors for Postoperative Mortality After Colorectal Surgery in 1049 Patients

Multivariate analysis found that only the same 4 independent factors, which were previously used for calculating AFC score, remain still independent factors in the present validation sample (Table 2): emergency surgery [P < 0.0001; OR = 7.638 (95% CI, 3.759–15.520)], body weight loss >10% [P < 0.0001; OR = 9.776 (95% CI, 4.848–19.714)], neurologic comorbidity [P = 0.011; OR = 2.850 (95% CI, 1.271–6.393)], and age older than 70 years [P = 0.0002; OR = 4.862 (95% CI, 2.090–11.310)].

TABLE 2. Multivariate Analysis of the Risk Factors Used in the AFC Score

Table 3 shows the variations of mortality rates and OR according to the AFC score value (ie, number of factors presents in the patient). The risk of death ranged from 0.5% to 1.6% when there was no or only one factor, it was 7% when 2 factors were present, nearly 47% when 3 factors are present, and as high as 70% when all the 4 factors were present. The area under the curve was 0.89 for the AFC score (Fig. 1). This value was higher than the area under ROC calculated for the ASA score, which was equal to 0.82 in the same sample. The Glasgow score completely failed to discriminate patients because a very high proportion of patients in both groups had a maximal value of the score.

TABLE 3. AFC Score: Variation of the Mortality Rate According to the Absence or Presence of the Four Risk Factors

FIGURE 1. ROC curve of the sensitivity and specificity of the AFC score. The under the curve area was 0.89.

Accuracy of a logistic model based on AFC score for mortality prediction was very good as shown in Table 4. This was also evidenced by the Hosmer-Lemeshow test, which was nonsignificant (P = 0.3669), indicating lack of deviation between the model and observed rates.

TABLE 4. Postoperative Mortality Rate Predicted by a Logistic Model Based on the AFC Score vs. Observed Postoperative Mortality

The separation parameter PSEP (see PATIENTS AND METHODS for definition) calculated as the difference between the mortality rate in the group with the worst prognosis category based on AFC score (ie, 70% for score = 4) and the mortality rate in the group with the best prognosis category based on AFC score (ie, 0.5% for score = 0) was equal to 69.5%. This value is higher than that which can be calculated from the results obtained in the sample used for building the AFC score, which was equal to 49.5%⁸ (Table 2). This indicates that the discriminatory capacity of the AFC score is fully maintained in this external validation sample.

Postoperative Morbidity

Postoperative morbidity was observed in 239 patients (23%). Complication rates were as follows: wound 8% (including wound dehiscence, wound infection, wound hematoma); clinical anastomotic dehiscence 5% (of which 42% of them underwent reoperation), postoperative hemorrhage (3%), and cardiorespiratory tract complications (7%). Reoperation was necessary in 42 patients (4%).

Mean hospital stay was 16 ± 13 days (range, 3–173 days).

DISCUSSION

This new French prospective multicenter study, including more than 1000 patients, showed that the AFC score was an accurate and reliable predictor of postoperative mortality after colorectal resection for cancer or diverticulitis. To our knowledge, this is the first French study that cross-validated one scoring system externally by applying the AFC score to a population separate from that used to develop the AFC score.

An external validation is a crucial step for building a prognosis score. Indeed, many prognostic scores suffer from an overoptimism about their discriminatory capacity. The consequence is that the discriminatory capacity initially found in the sample used to build the score is not reproduced in subsequent studies. Different methods for internal validation of parameter calibration allow us to reduce this problem but not to avoid it completely. In the present study, we confirm the high differences between mortality rates among the different values of the AFC score. In a formal manner, we used the PSEP statistics recently proposed, which allow us to quantify the capacity of the score to discriminate between good and bad prognosis.¹⁴ As noted by Altman and Royston, it is frequent that PSEP value is high in the first study used to propose the score (ie, indicating a good capacity of the score to discriminate between good and bad prognosis) but decreases in a second study for an external validation sample, indicating the over-optimism of the first study due to data driven model selection.¹⁴ This is clearly not the case for the AFC score since the PSEP value is even higher in the present validating study than in the study used to build the AFC score.

The present study represents the first attempt to validate a reliable scoring system (the AFC score) for use specifically in colorectal surgery. Another interesting aspect of the AFC score is that a very large number of centers have participated in the building or in the validating process, thus also explaining the good generalizibility of the score properties evidenced in the present study. As the first French multicenter study, colorectal cancer and diverticulitis were only included in this study because they were the most frequent indications for colorectal resection in general or private centers.

The main advantage of the AFC score is that it allows a given surgeon to predict postoperative death before surgical procedures, which then can help in counseling patients and families to have realistic expectations. Furthermore, it can be used to make easy comparison on the quality of care between institutions or between surgeons within institution.¹⁵

Most operative scoring systems have been proposed for the purpose of assessing risk for an individual surgical patient or as a means of determining which patients may benefit from additional preoperative optimization.¹⁶ They are based on complex mathematical formulas (ie, POSSUM, P-POSSUM, and CR-POSSUM) and could be used in clinical research but rarely in daily clinical practice.¹⁶ Although the POSSUM scoring systems have been validated in patients undergoing general, colorectal, and vascular surgery,^16–18 several studies have reported that POSSUM and P-POSSUM scoring systems overpredicted mortality¹⁹ and morbidity,^20,21 particularly in young patients and elective colorectal procedures.²² In this study, we did not use POSSUM and P-POSSUM scoring systems because we reported previously that both of these scoring systems significantly overpredicted the mortality compared with the AFC score.⁹ Recently, the newly developed CR-POSSUM for colorectal surgery was found to have better calibration and discrimination than the existing POSSUM and P-POSSUM scoring systems.⁶ However, in the calculation of predicted mortality based on CR-POSSUM, 6 physiologic factors and 4 operative severity factors were scored based on the data for the POSSUM system. Furthermore, CR-POSSSUM is calculated with a complex mathematic formula (Log_e [R_CR/(1 − R_CR)] = [0.338 × (CR physiologic score)] + [0.308 × (CR − operative score)] − 9.167 where R_CR is the risk of mortality), and is difficult to use in daily practice. Furthermore, Law et al have reported that not only POSSUM and P-POSSUM, but also CR-POSSUM overestimated mortality and morbidity in patients who underwent laparoscopic colorectal resection.²³ Finally, the validity of the CR-POSSUM model needs to be validated externally in a population separate from that used to develop the CR-POSSUM scoring system. For all these reasons, we believed that it was not mandatory to compare our AFC score with CR-POSSUM during our external validation of the AFC score.

Operative mortality is an objective measure of health care that can easily be measured as in-hospital or 30-day mortality. In the present study, in-hospital mortality was used because it can be recorded with greater accuracy than death after discharge from hospital. However, with the current tendency toward early discharge, after even major surgery, a combination of in-hospital and 30-day operative mortality may be necessary in the future.

Among the AFC score, 3 of these factors (emergency, age older than 70 years, and neurologic disease history) cannot be modified preoperatively, but we foresee that expected mortality could be reduced by acting (in elective surgery) on the fourth item, which is a weight loss >10%, reflecting the preoperative malnutrition of the patient. Malnutrition appears as a risk factor of postoperative mortality. Malnutrition may be evaluated, either by a weight loss of more than 10% within the past 6 months such as in our study, or by serum albumin level such as described by Longo et al.²⁴ Gibbs et al have reported that serum albumin level was a strong predictor of mortality, and it is the best prognostic indicator of nutritional status.²⁵ However, the drawback of using specialized investigations such as albumin, prealbumin, or liver function tests is that, although they may be readily, prospectively collected in elective situations, this is not always the case in an emergency setting when data collection and score generation may be incomplete, leading to incorrect assessment of operative risk. A recent prospective randomized study has reported that the administration of a supplemented diet before and after surgery was beneficial to outcome in malnourished patients with cancer undergoing major elective surgery.²⁶ Perioperative treatment with immunonutrition seemed to be the best strategy to reduce complications and length of hospital stay.

CONCLUSION

This prospective study was conducted on a new population of patients with the aim of validating a predictive score of mortality after colorectal surgery, previously proposed.⁹ The present study demonstrated that the AFC score is an accurate and reliable predictor of postoperative mortality after colorectal surgery. Because it is based on only 4 risk factors, the AFC score can be used in daily practice.

ACKNOWLEDGMENTS

Participating center: C. Adamsky (Belfort), A. Alves (Paris), J. M. Andreu (Marseille), G. Angelvin-Bonnetty (Avignon), Y. Anselm (Strasbourg), L. Arnalsteen (Lille), J. P. Arnaud (Angers), A. Ayave (Vandoeuvre-lès-Nancy), P. Baillet(Montmorency), P. Balandraud (Marseille), M. Balkanski (Lons-le-Saunier), B. Baranger (Paris), P. Baudot (Reims), A. Bellouard (Olivet), S. Berdah (Marseille), A. Berger (Paris), D. Beyrne (Avignon), P. Boissel (Vandoeuvre-lès-Nancy), A. Bourgeon (Nice), R. Boutboul (Salon-de-Provence), P. Y. Bouteloup (Saint-Grégoire), L. Bresler (Vandoeuvre-lès-Nancy), P. Bruant (Hagueneau), F. Brunetti (Créteil), J. L. Caillot (Pierre-Bénite), G. Carbonnel (Mende), J. L. Cardin (Laval), N. Cardin (Grenoble), T. Carrelet (Apt), P. Chenet (Annecy), S. Chokairi (Ussel), G. Chorvat (Romans), F. Codinach (Verdun), D. Collet (Pessac), P. Cubertafond (Limoges), J. M. Cucchi (Pessac), A. Dabrowski (Seclin), M. de Cooman (Bruxelles), D. Degroote (Boulogne-sur-Mer), C. Delteil (Châteauroux), C. Denet (Paris), B. Descottes (Limoges), N. Destrumelle (Annecy), J. Emer (Avignon), O. Emungania (Salon-de-Provence), J. P. Favre (Dijon), J. L. Faucheron (Grenoble), P. Ferrandis (Ussel), O. Firtion (Sarreguemines), J. P. Flament (Reims), J. Fraisse (Dijon), J. Y. François (Montbéliard), P. Gabelle (Grenoble), A. Gainant (Limoges), R. Gebbala (Limoges), J. F. Gillion (Thiais), B. Goffre (Pessac), M. Guillaume (Fontenay-le-Comte), P. Guivarch (Castres), P. Herbière (Albi), J. P. Houze (Belfort), G. Izard (Tarbes), S. Kaafarani (Paris), I. Kalouche (Verdun), A. Kartheuser (Belgique), A. Lababidi (Nogent-le Rotrou), A. Lacroix (Auch), N. Lamfichekh (Montbéliard), M. Lampert (Strasbourg), C. Lang (Vesoul), J. le Borgne (Nantes), J. R. Legros (Rouen), P. A. Lehur (Nantes), J. P. Letoquart (Quimper), S. le Toquin (Caen), N. Lévy-Chazal (Reims), C. Mangia (Montmorency), G. Mantion (Besançon), C. Mariette (Lille), B. Masson (Pessac), J. Maurel (Caen), V. Maylin (Chambéry), F. Ménégaux (Paris), C. Meyer (Strasbourg), P. Meyer (Genève), G. Millon (Lons-le-Saunier), J. Paineau (Nantes), J. Palot (Reims), Y. Panis (Paris), M. Papillon (Lyon), Y. Parc (Paris), A. Paupert (Saint-Julien-en-Genevois), E. Pelissier (Besançon), P. Pessaux (Angers), P. Peyrat (Lille), A. Picard (Belfort), J. P. Porta (Desertines), C. Proye (Lille), J. P. Pujol (Saint-Cloud), J. B. Putinier (Grenoble), E. Rasolofo (Avignon), P. Rat (Dijon), A. Rault (Pessac), A. Rey (Bourg-en-Péage), D. Rio (Vannes), M. Rivoire (Lyon), D. Robert (Beuvry), S. Rohr (Strasbourg), P. Rousseau (Mende), E. Rullier (Bordeaux), C. Runser (Montbéliard), A. Sa Cunha (Pessac), M. S. Sbai Idrissi (Montmorency), G. Schmitt (Reims), H. Sebbag (Salon-de-Provence), P. Segol (Caen), S. Serhal (Liban), P. Skawinski (Bergerac), K. Slim (Clermont-Ferrand), M. Soualmi (Saint-Julien-en-Genevois), E. Tarla (Cannes), P. Ténière (Rouen), J. Thanwerdas, E. Tiret (Paris), J. M. Tortuyaux (Vandoeuvre-lès-Nancy), J. Touchet (Avignon), O. Towa (Colmar), P. Trancart (Cahors), G. Traverse (Vesoul), J. P. Triboulet (Lille), J. R. Tubiana (Elbeuf), P. Valleur (Paris), B. Weill (Saint-Jean-de-Maurienne), A. Zahredine (Boulogne-sur-Mer), J. Zgheib (Dijon).