Application of the Parsonnet scoring system for a Canadian cardiac surgery program

Abstract

BACKGROUND:

In the past two decades, cases involving patients requiring cardiac surgery have become more complex, presenting with more comorbidities. Outcome analysis has become very important in assessing the quality of cardiac surgical care in these patients. The latest version of the Parsonnet scoring system was developed in 2000 and is the most recent system available.

OBJECTIVE:

To evaluate the accuracy of the Parsonnet scoring system in a major Canadian university-based cardiac surgery centre with a population of high-risk patients.

METHODS:

Data on 4883 consecutive patients operated on between 2000 and 2005 were prospectively collected, and a standardized mortality rate was calculated using the Parsonnet score as the ratio of observed deaths to expected deaths. Analyses were conducted on the whole group and on subgroups, based on Parsonnet score distribution quantiles, age and surgery status.

RESULTS:

The mean Parsonnet score was 18.8±13.7 (range 0 to 83). The overall mortality rate was 6.4%. The overall standardized mortality ratio was 0.52 (95% CI 0.420 to 0.568), which was statistically significant (P=0.01). The observed mortality rate was significantly lower than expected in all categories.

CONCLUSIONS:

Despite more complex cases with multiple comorbidities, the results of cardiac surgery in a Canadian university hospital show better results than expected when using the Parsonnet score.

The past two decades have seen a significant change in the profile of patients who undergo cardiac surgery. The number of patients undergoing coronary artery bypass grafting (CABG) surgery is decreasing, but the proportion of patients requiring valve surgery is increasing. Moreover, cardiac surgery is now offered to high-risk patients who have several comorbidities. For these patients, surgical treatment was not an option in the previous decade. In this context, it is important that the evaluation of patient outcomes takes into account the overall patient risk profile. Quality management should not be based on crude assessments of mortality or complications, but rather on estimates of these outcomes that are adjusted for the baseline state and overall risk profile of the patient population. Failure to take baseline risk into consideration results in biased estimates of outcomes and poor quality assessments for institutions that are providing high-level, state-of-the-art care to high-risk patients. Patient registries are a valuable tool in conducting such independent reviews. In Canada, there is limited implementation of provincial databases in cardiac surgery with risk-adjustment algorithms, such as the one proposed by the Cardiac Care Network of Ontario (1). The province of Quebec, to date, does not have a provincial database for cardiac surgery.

Many scoring systems have been designed to predict mortality rate among cardiac surgery patients (1–6). Most of these scoring systems have been designed to assess CABG patients, but they have been widely applied to patients undergoing valvular heart operations. The EuroScore (3), The Society of Thoracic Surgeons database score (5,6) and the Parsonnet score (2) have been used to predict the risk of short-term mortality on the basis of preoperative patient characteristics. Evaluation of these risk models has shown that they are valid, especially in the low- or intermediate-risk patients. The predictive power of these models, however, becomes weaker for the higher-risk categories because of the smaller sample sizes.

The Division of Cardiac Surgery at the McGill University Health Centre (Montreal, Quebec) adopted the latest version of the Parsonnet scoring system after it was published in 2000 (2). This version was based on 37 parameters that are used to predict the perioperative mortality ratio of cardiac surgery patients. The purpose of the present study was to assess quality of care as measured by standardized mortality ratios (SMRs) based on quantifiable risk estimated by the Parsonnet scoring system, in a university-based cardiac surgery centre. This assessment is particularly important given the increasing prevalence of high risk cardiac surgery patients in our institution.

METHODS

Data were collected prospectively over a five-year period (2000 to 2005) and entered into the McGill University cardiac surgery registry. All patients undergoing cardiac surgery at the McGill University Health Centre are entered into the registry. Data were abstracted from patient charts and were entered into the registry by cardiac surgery nurses who were trained to abstract and enter data. All data were reviewed by the treating surgeon for quality assessment and final record closure. A Parsonnet score was calculated for all patients undergoing elective, urgent or emergency cardiac surgery at the two McGill University Health Centre sites, the Royal Victoria Hospital and the Montreal General Hospital (both in Montreal, Quebec). Individual patient Parsonnet scores were rounded to the nearest half-integer. Thus, the total score range was 0 to 100. Excluded were patients undergoing heart or heart-lung transplantation, as well as ascending, arch or thoracic aortic surgery, ventricular assist device implantation and adult congenital cardiac surgery. This is because the complexity of the procedures currently does not allow precise measurement of the risks associated with the operations. A total of 4883 patients were included in the present study.

Perioperative mortality in the present study was defined as proposed by The Society of Thoracic Surgeons, and included “death from any cause during or after operation within 30 days if the patient is discharged or within any interval if the patient is not discharged” (7). Moreover, for the purpose of the present study, inter-hospital transfers were not considered to be patient discharge.

Descriptive statistics were produced for all relevant variables. SMRs were calculated by indirect standardization using the Parsonnet score. For each patient, risk of mortality was estimated using the logistic regression parameters reported for the Parsonnet score (2). Therefore, for each patient the odds of death (O_death) = e^c+γ, where c is the constant and γ = β₁X₁+β₂X₂+…β_nX_n. β is the logistic regression coefficient for each variable and X is the value of the variable for the patient. The probability of death for each patient was estimated using the estimated O_death as O_death/(1+O_death). The sum of the individual patient probabilities of death was used as the estimate of the number of expected deaths. SMRs were calculated as the ratio of observed deaths to expected deaths. The 95% CI for the SMR estimates were calculated according to the methods developed by Breslow and Day (8). An SMR of 1.0 indicates no difference between the observed and expected mortality rates. This suggests quality of care that is equivalent to the established standards. An SMR of less than 1.0 indicates quality of care that is superior to the established standard of care, and an SMR of greater than 1.0 indicates quality of care that is inferior to the standards of care. SMR analyses were conducted for the study sample as a whole, and for patient subgroups defined according to Parsonnet score distribution quantiles, age (younger than 70 years, 70 to 75 years, 76 to 79 years, and 80 years or older) and surgery status (elective, emergency and urgent). For these subgroups, mortality risk was estimated from the Parsonnet score without consideration of the variance in the risk associated with each subgroup. These stratified analyses were conducted to observe variation in the quality of care between clinically important patient groups. In the article published by Bernstein and Parsonnet (2), the maximum Parsonnet score was 50.0. However, the actual possible range of the Parsonnet score is between 0 and 100. In the current study, patients with Parsonnet scores greater than 50.0 were included in the analysis. This is acceptable, because the estimation of mortality risk is not limited by the observed score range of the original article, but rather by the actual possible range of the score. The predictive accuracy of the score is based on the validity of the parameter estimates for each variable included in the model. It follows that a score of any value can be achieved through various combinations of different variables, not only a limited set of variables. Therefore, scores beyond the reported range are possible and should have the same predictive validity as lower scores. Therefore, the estimates of the mortality risk for patients with scores greater than 50.0 remain valid, and these patients were included in the analysis. As a reference, however, patients with Parsonnet scores greater than 50.0 were analyzed as a separate subgroup.

RESULTS

There were 4883 patients included in the present study, 56.3% of whom were male. The mean patient age was 65.6±11.0 years, with a range of 18 to 94 years and a median of 67 years. Approximately 20% of the patients were older than 75 years of age, and 7% were older than 80 years of age. A total of 3271 patients (67%) underwent isolated CABG surgery, while the others underwent valvular surgery or combined procedures. Seventy-seven per cent of patients with a score of less than 30 underwent isolated CABG surgery, while only 33% of patients with scores of 30 or higher underwent CABG surgery alone. The prevalence of comorbid conditions in the study cohort is summarized in Table 1. These results show that the majority of patients had a history of hypertension (74.9%), followed by congestive heart failure (37.3%), diabetes (30.9%), ejection fraction between 30% and 49% (30.9%), left main coronary artery disease (21.6%) and pulmonary hypertension (15.1%). Figure 1 demonstrates the distribution of Parsonnet scores for the study population. The mean Parsonnet score for the study cohort was 18.8±13.7, with a range of 0 to 83 and a median of 16. Operative status was urgent for 2672 patients (54.7%), elective for 1785 patients (36.6%) and emergent for 426 patients (8.7%). The overall mortality rate was 6.37% (n=311).

TABLE 1.

Prevalence of comorbid conditions (n=4883)

Condition	n (%)
Hypertension	3657 (74.9)
Congestive heart failure	1821 (37.3)
Diabetes	1509 (30.9)
Ejection fraction 30% to 49%	1509 (30.9)
Left main coronary artery disease	1055 (21.6)
Pulmonary hypertension	737 (15.1)
Renal failure	615 (12.6)
Mitral valve	620 (12.7)
Valve and aortocoronary bypass	591 (12.1)
Chronic obstructive pulmonary disease	542 (11.1)
Morbid obesity	493 (10.1)
Ejection fraction <30%	483 (9.9)
Severe peripheral vascular disease	474 (9.7)
Aortic valve	469 (9.6)
Severe neurological disorder	293 (6.0)
Preoperative intra-aortic balloon pump	269 (5.5)
Cardiogenic shock	249 (5.1)
Carotid disease	171 (3.5)
Asthma	146 (3.0)
Dialysis dependency	146 (3.0)
Active endocarditits	88 (1.8)
Tricuspid valve	78 (1.6)
Cirrhosis	44 (0.9)
Idiopathic thrombocytopenia	19 (0.3)
Pacemaker	24 (0.5)
Treated endocarditits	24 (0.5)

Figure 1).

Patient distribution in each Parsonnet score (n=4883)

Table 2 summarizes the SMR analyses for the study sample and relevant patient subgroups. These results show that there were a total of 311 deaths (6.37%), while 594 deaths (12.16%) were expected according to indirect standardization. The overall SMR was 0.52 (95% CI 0.420 to 0.568). This indicates that the observed mortality rate was 48% less than that expected. The overall SMR was statistically significant (P=0.001).

TABLE 2.

Standardized mortality ratio analyses

	Stratum	Patients, n	Mortality		Standardized mortality ratio		P
			Observed, n (%)	Expected, n (%)*	Estimate	95% CI
Total	All	4883	311 (6.37)	594 (12.16)	0.52	0.424–0.568	0.001
Parsonnet score (categories)	0–6.9	951	4 (0.4)	20 (2.2)	0.20	0.069–0.583	0.001
	7.0–12.9	744	7 (0.9)	25 (3.4)	0.28	0.122–0.643	0.001
	13.0–16.9	831	17 (2.0)	39 (4.7)	0.44	0.249–0.764	0.001
	17.0–22.9	749	27 (3.6)	56 (7.5)	0.48	0.308–0.755	0.001
	23.0–32.9	829	53 (6.4)	117 (14.1)	0.45	0.332–0.618	0.001
	33.0–50.9	630	128 (20.3)	226 (35.9)	0.57	0.476–0.692	0.001
	≥51	149	75 (50.3)	110 (73.7)	0.68	0.566–0.821	0.001
Age (years)	<70	3032	111 (3.66)	273 (9.02)	0.41	0.304–0.485	0.001
	70–75	881	77 (8.74)	116 (13.2)	0.66	0.460–0.866	0.001
	76–79	535	55 (10.28)	99 (18.53)	0.55	0.349–0.730	0.001
	≥80	435	68 (15.63)	105 (24.13)	0.65	0.409–0.829	0.001
Surgical status	Elective	1785	54 (3.03)	169 (9.47)	0.32	0.215–0.413	0.001
	Emergent	426	92 (21.6)	115 (27.08)	0.80	0.537–1.033	0.080
	Urgent	2672	165 (6.18)	309 (11.58)	0.53	0.424–0.568	0.001

*Calculated as the average individual predicted mortality risk, as estimated by the Parsonnet score for each patient in the corresponding stratum

Figure 2 shows the linear regression modelling for the observed and expected mortality rates as a function of the Parsonnet score. These results show that the observed mortality rate was significantly lower than the expected rate according to indirect standardization to the Parsonnet model. Linear regression analyses produced a parameter estimate of 0.0082% per Parsonnet score unit for the expected mortality rate compared with 0.0052% per unit for the observed mortality rate. These two parameter estimates are significantly different (P<0.001) and indicate an overall adjusted SMR of 0.63 (P<0.001).

Figure 2).

Linear regression model comparing the observed mortality with expected mortality in study patients

Stratification by Parsonnet score showed that the observed mortality rate was significantly lower than expected for all categories, as indicated by SMR estimates that were statistically significantly lower than unity (P=0.001).

Figure 3 shows the actual Parsonnet-predicted mortality risk rate compared with the observed McGill University Health Centre patient mortality rate for all score categories. The solid line is based on a multivariate model for the observed mortality rate as a function of the Parsonnet score. The McGill University Health Centre mortality rate was lower for all categories of scores.

Figure 3).

Comparison of Parsonnet curve to observed mortality curve at the McGill University Health Centre ([MUHC] Montreal, Quebec). The multivariate curve was derived from the observed mortality curve in 4883 patients through the curve-fit model of Excel software (Microsoft Windows Office 2003; Microsoft Inc, USA)

Stratification by age also showed significantly lower observed versus expected mortality rates, as shown by the significant SMR estimates (P=0.001). The lowest SMR estimate was observed for those patients who were younger than 70 years of age, followed by the estimate for patients between 76 and 79 years of age. Higher SMR estimates were observed for patients in the 70 to 75 years old and 80 years or older age groups.

Patients undergoing elective surgery had significantly lower SMR rates than patients undergoing emergent or urgent surgery. The SMR estimate for elective and urgent cases was statistically significantly lower than unity (P=0.001); however, the estimate for emergent patients was lower than unity, but only approached statistical significance (P=0.080). The latter observation may be due to the smaller number of patients in the emergent surgery group.

DISCUSSION

In the current study, the Parsonnet score was used to produce estimates of expected pericardiac surgery mortality as a function of baseline patient comorbidity. The Parsonnet score model was recently developed using data from a large cohort of patients undergoing cardiac surgery procedures in 10 New Jersey centres during 1994 and 1995 (2). Other methods have been developed primarily for CABG surgery patients, while the Parsonnet score has been based on 10,703 patients undergoing CABG, valve replacement or repair, or a combination of procedures. The advantage of the Parsonnet score is that the parameters representing risk factors are easily and objectively measurable, and have high content validity that is recognized and accepted by the majority of clinicians. As opposed to other scoring methods, variables that are based completely or even partly on subjective assessment, such as unstable angina, operative priority or diffuseness of disease, are not included in this model. Outcomes after CABG surgery have been extensively studied (9–14), and many scoring systems have been developed for these patients; their use has been extended to other types of cardiac operations. As the number of CABG procedures rapidly declines and the number of valvular or combined operations rises, it is important to use a scoring system that is applicable to all patients.

One of the inherent problems with most predictive scoring systems is the very broad CI for higher-score categories. This is due to the smaller numbers of patients and the higher variability in these higher categories. Most scoring systems in use today were developed using cohorts of patients in which isolated CABG was the predominant procedure. The paper by Parsonnet and Bernstein (2) does not mention the proportion of CABG surgery compared with other procedures. Our study clearly documented that higher-risk patient cohorts consist mostly of patients undergoing procedures other than isolated CABG surgery (67% of patients with scores of 30 or greater underwent other procedures). The higher variability of procedures may partially explain the lower predictive power of the model for scores of 30 or greater.

Risk stratification models have been developed for valve surgery (4,15,16). Nowicki et al (16) suggested two models for risk stratification of patients undergoing either aortic or mitral valve surgery. These mathematical models were highly accurate in predicting in-hospital mortality. The area under the receiver operating characteristic curve was 0.75 (95% CI 0.72 to 0.77) for aortic valve patients and 0.79 (95% CI 0.76 to 0.81) for mitral valve patients.

However, the Parsonnet scoring system includes only patient characteristics and comorbid conditions that can be easily ascertained before surgery, and may appear better suited to an increasingly diversified population of patients in terms of procedures and comorbidities.

The model has also been used to preoperatively identify patients who are likely to spend longer than 24 h in the intensive care unit (17). In another study (18), a Parsonnet score of greater than 20 strongly predicted readmission to the intensive care unit in fast-tracked patients (OR 3.08, 95% CI 1.43 to 6.69). Such a model can therefore serve multiple purposes in the administration of a busy cardiac surgery service in an era of scant resources.

This was a retrospective study of registry data obtained during a five-year period from a large, Canadian, university-based cardiac surgery centre. A review of patient profiles revealed that the study cohort had a relatively high-risk baseline profile. This is evident by increased age and the high proportion of patients older than 75 years of age. Thirty-three per cent of patients in this cohort underwent procedures other than isolated CABG surgery. This declining proportion of CABG operations over other procedures has been observed around the world in recent years and should be taken into consideration when assessing outcomes of cardiac surgery as a whole. In addition, our cohort had a high prevalence of clinically important comorbid conditions, including congestive heart failure and diabetes, as one would expect from general, multispecialty tertiary centres performing cardiac surgery. Moreover, a high-volume heart failure and transplantation program attracts a large number of patients who are now offered conventional surgical options.

The results of our study have shown that despite the increased baseline risk, the observed mortality rate in our cohort was consistently significantly lower than that expected by standardization with the Parsonnet score cohort. This was observed for moderate-risk and high-risk patients. The higher patient risk profile in our institution during the study period has contributed, through volume-performance association, to the improved outcomes of our high-risk patients. It is safe to presume that the Parsonnet scoring system can be applied to our cohort of patients undergoing cardiac surgery in the current era, as well as in the Canadian health care system. It can be used to measure quality of care and plan for resource use.