Role of volume outcome data in assuring quality in HPB surgery

Abstract

Many studies have shown an association between both surgeon and hospital operative procedure volumes and outcomes, particularly operative mortality. It is also recognized that volume is only one of a number of factors, including 1) surgeon training and experience, and 2) hospital resources, organization, and processes of care, which can also influence outcomes. The Surgical Oncology Program at Cancer Care Ontario has included hospital volumes in a set of standards for the conduct of major pancreatic cancer surgery, along with recommendations for surgeon training and hospital resources, organization, support services, and processes of care to encourage regionalization of major HPB surgery. Cooperation with these recommendations was encouraged by the public reporting of mortality data and by an educational program directed at both surgeons and senior administrators in Ontario hospitals with the support of the provincial health ministry. The provincial mortality rate from major pancreatic cancer surgery has decreased by more than 50% since the introduction of this program.

Background

The association between case volume and improved outcomes is playing an increasing role as an indicator of the quality of surgical practice in a variety of settings. This association has been studied between both surgeon and hospital operative procedure volumes and a range of outcomes including operative mortality, complication rates, hospital costs, and long-term survival ¹,². Most of these studies have shown a positive relationship between increasing volume and better outcomes, although there are exceptions. The most consistent relationship has been demonstrated between volume and postoperative mortality. The association is stronger for hospital than for surgeon volume, and the most consistent relationship is seen for operations that are complex and carry high risk. Although there are extensive data supporting this volume outcome relationship, there are some limitations to the conclusions that can be drawn from these data. There are numerous individual hospitals and surgeons that are exceptions to the usual pattern, many of the quoted publications refer to hospital data that may not represent current practice, and there is wide variation in the type of risk adjustment, if any, that was used in many of the reported studies.

A common problem in the existing reports is that the highest volume category reported tends to underestimate the actual procedure volumes, since that category is open ended and its reference point is the lowest volume in the range. This can be particularly problematic when these data are used to estimate volume thresholds that are being recommended to achieve desired outcomes. Table I shows the volume thresholds and corresponding actual mean numbers of cases per year in the highest volume range in eight hospital volume outcome studies that examined the relationship of postoperative mortality to case volume following major pancreatic resection ¹,³,⁴,⁵,⁶,⁷,⁸,⁹. The actual mean number of cases per year in the highest volume category in these studies varied from 1.2 to 5.0 times the volume threshold reported. Since the data in the literature suggest that the volume outcome relationship tends to be consistent over a wide range of volumes, it is more likely that good outcomes observed in these high volume ranges are related to the mean volumes observed than to the threshold volumes for the highest volume categories. This has not been considered in the past when recommendations have been made regarding volume thresholds being established for quality improvement purposes.

Table I. Hospital volume/outcome studies of mortality following pancreas resection.

	Highest volume category
	Cases per year
Reference	Threshold	Mean	Mortality rate (%)
Edge et al. 1993 [3]	>2	8.4	5.1
Lieberman et al. 1995 [4]	>10.1	23.4	5.5
Glasgow & Mulvihill 1996 [5]	>10	14.3	3.5
Imperato et al. 1996 [6]	>6.3	17.2	2.2
Gordon et al. 1998 [7]	>20	28	1.8
Sosa et al. 1998 [8]	>20	88	0.8
Simunovic et al. 1999 [9]	>6	17.2	3.4
Birkmeyer et al. 2001 [10]	>16	80.0	3.8

It is also generally agreed now that volume alone is not enough to explain variation in outcomes between hospitals or between surgeons. There are many other factors that can contribute to improved outcomes that have been observed to be associated with increased volumes. These can be classified as factors that are either related to surgeons themselves or to the systems in which surgeons work. At the surgeon level, the intuitive hypothesis is that ‘practice makes perfect’; however, any occasional golfer can testify that practice alone is not enough reach a high level of performance. A more appropriate version of that hypothesis might be ‘perfect practice makes perfect’, which fits better with the modern concepts of surgical training, including supervision, performance standards, objective feedback, and subjective self-assessment. In addition to the effects of training and mentoring, there are other considerations including natural ability and motivation that may play an important role in determining surgeon performance, but these are much more difficult to measure.

When considering the role of the system in which a surgeon works, one can examine both its structure and its processes of care. Structure includes physical and human resources necessary to carry out the specific service required. These include things like diagnostic imaging, pathology, OR and ICU capability, and the support of other medical and related specialists who are also trained to a high level of competence. Sufficient numbers of patients (volume) to develop and maintain the expertise required of all the involved health care workers is just one of the structural elements that can contribute to good outcomes.

Processes of care are the wide variety of transactions – both complex and simple – between a patient and healthcare providers, between different providers, and between a patient and his/her health care environment. These interactions may be provider planned and initiated, spontaneous, accidental, protocol- and evidence-based, or completely idiosyncratic. Because modern health care is so complex, with so many interactions occurring in an individual patient, the opportunity for error is high, and the need for system design around evidence-informed processes of care is paramount. Quality improvement initiatives need to include a careful examination of all relevant processes of care in addition to structural and training issues to be successful.

Volume alone is just a surrogate predictor of quality, but from the evidence, it is a pretty good surrogate, and has the benefit of being easily measurable. The relationship of volume to performance has achieved sufficient credibility to be used in the marketing of health services, and as a requirement for preferred provider status, for example by the Leapfrog Group in the United States ¹⁰. Case volume, expressed as population served, has also been used as a requirement in regional health service planning for HPB surgery by the National Health Service in the UK ¹¹.

This paper will outline how procedure volume is being used as part of a process of quality improvement in surgery in the province of Ontario, Canada, along with a number of other elements including surgeon training and experience, hospital structure and processes of care, and regional and provincial planning and policy making.

Quality improvement in cancer surgery in Ontario

Cancer Care Ontario (CCO) is the provincial agency responsible for cancer services for a population of 12.5 million people in Ontario, Canada. The CCO Surgical Oncology Program, created in 2001, has developed a Quality Improvement Program based on a) development of guidelines and standards; b) measurement of performance through the use of indicators which are reported publicly; and c) implementation of QI initiatives using a knowledge translation strategy based on Communities of Practice in individual surgical specialties.

The overall goal of the Surgical Oncology Program is to develop a cancer surgery care delivery system that is of high quality, patient-centered, assessable, interdisciplinary, and integrated. This is being done through the development of regional networks of care in which ‘standard’ care is more widely distributed and complex or uncommon care will be delivered in regional or provincial or specialized centers. The quality expectations are the same whether the centers are large or small. Patients will thus able to receive most of their care close to home, but may have to travel to access high quality complex care.

Public reporting of quality indicators was initiated in 2004 with the introduction of the Cancer Services Quality Index ¹². This is a public report on the Cancer Care Ontario website of 26 indicators covering the spectrum of cancer prevention, screening, treatment, and outcomes. Included are surgery-related indicators such as hospital procedure volumes, mortality rates, waiting times, utilization rates, lymph node retrieval, and margins status. Considerable information is also available to the Surgical Oncology Program through national and provincial databases to measure other process and outcomes that may not be reported publicly.

Volume and outcomes in pancreatic cancer

In 1997 a study of the relationship of hospital volume and postoperative mortality following radical pancreatic resection (Whipple or total pancreatectomy) was carried out in Ontario ⁹. A preliminary analysis of the data was in keeping with that from other jurisdictions and showed a provincial mortality rate of 10.2%, with a higher mortality rate in low volume hospitals and a lower mortality rate in high volume hospitals. Because the study had been supported by the Institute of Clinical Evaluative Sciences, a provincially funded research body, there was an obligation to make the data public. However, before this was done, an expert panel was created to develop a set of standards for the conduct of major pancreatic surgery so that the release of this information would be accompanied by a reassurance to the public that steps were being taken to correct the problems identified. This expert panel produced a document entitled ‘Criteria for the Delivery of Pancreatic Cancer Surgery’, which outlined recommendations for: the formal training and surgeon experience required for the conduct of HPB surgery, the necessary hospital resources, organization, and infrastructure, and for the volumes of both major pancreatic surgery (at least 10 cases per year) and total HPB surgery (at least 25 cases per year) considered to be required for optimum outcomes ¹³. A benchmark for postoperative mortality of < 5% was proposed.

The public release of the hospital mortality data was accompanied by articles in the newspapers and the expected negative reaction from some hospitals and surgeons. However, when the standards were then distributed many hospitals made changes in the management of pancreatic cancer, including some that disallowed some or all of the surgeons to do those operations, and others that reorganized services to better meet the standards that had been recommended. Strategies for continuing education of surgeons included presentations at scientific meetings and informal contacts through communities of practice. A qualitative review by questionnaire was carried out in 2001, and 57 of 92 hospitals responded. One quarter had made changes to comply with the recommendations and half of the hospitals reported that at least one surgeon had stopped doing pancreatic surgery. A more detailed follow-up was carried out in 2005 and this showed that the number of these cases that were being treated in hospitals with volumes of 10 or more cases per year had increased from 17.8% in the early study to 60.8% in the years 2002–2004. The provincial 30-day case fatality rate (death in hospital or within 30 days) had decreased from 10.2% to 4.5%.

Figure 1 shows the redistribution of major pancreatic surgery volume between the two time periods with a reduction in the total number of hospitals involved from 72 to 28 and a shift in the case volume from low volume hospitals (<10/year) to medium (10–20/year), and high volume (20 + /year) hospitals. Figure 2 shows that in addition to the decrease in the mean mortality rate across the province there was a decrease in the mortality rates in each of the volume categories. This is in keeping with the hypothesis that there were other factors involved in the overall improvement in outcomes in pancreatic surgery in addition to case volumes. Our assumption is that more attention was paid to the surgeon training and hospital structural and processes of care based on the new standards, and that public disclosure played an important role in motivating these changes. These changes in Ontario were unique to pancreatic cancer surgery. For example, standards for esophageal cancer surgery had not yet been developed nor had there been public disclosure of esophageal operative mortality rates. The remarkable decrease observed in the mortality rate from pancreatic surgery during this time period did not occur with esophageal surgery.

Figure 1. .

Pancreatic resection in Ontario: volume distribution 1988–96 and 2002–2004.

Figure 2. .

Pancreatic resection in Ontario: mortality rates 1988–96 and 2002–2004.

There were also efforts made in the Netherlands (population about 15.5 million) at about the same time to deal with the excess mortality from pancreatic cancer surgery in low volume hospitals ¹⁴. In a study of pancreatic resections from 1994 to 1998 the national postoperative mortality was reported as 10.7%, with a death rate of 16% in the lowest volume institutions and 1% in the highest. These data were presented at multiple meetings of surgeons in the Netherlands over the next few years. In a follow-up study reported in 2006 ¹⁵, the mortality rates and variation between low and high volumes hospitals in 2001–2003 remained essentially unchanged from the previous study. There was also little change in the proportion of patients being operated on in low volume institutions. One can speculate about various reasons for the current differences in case distribution and mortality between Ontario and the Netherlands, but the proactive approach of standard development and promotion along with public reporting of mortality outcomes would seem to be likely candidates.

Because almost 40% of the pancreatic cases in Ontario were still being treated in hospitals with volumes less than 10 cases per year, another expert panel was convened to review and update the pancreatic standards, this time expanding the scope to all HPB surgery. The resulting document entitled ‘Hepatic, Pancreatic and Biliary (HPB) Surgical Oncology Standards’ was completed in June 2006 and is available on the Cancer Care Ontario website ¹⁶. The surgeon and hospital requirements were similar to the previous set of standards except for a more explicit accountability requirement in the recommended administrative structure of a HPB service. The volume targets were increased to at least 20 major pancreatic and a total of 50 major HPB cases per year. These volumes would serve a population of approximately one million people based on current levels of activity. The benchmark operative mortality for pancreatic resection was left at <5% and for anatomical liver resection was set at <3%. This document along with a similar set of standards for thoracic cancer surgery and for multidisciplinary cancer conferences is currently being used to inform the planning process being carried out by new regional health authorities that have been established by the Ontario government. Additional cancer surgery standards and guidelines are currently being developed in CCO by the Surgical Oncology Program along with the Program in Evidenced Based Care ¹⁷.

Conclusion

Hospital and surgeon case volumes are among a number of factors that can influence outcomes following major surgical procedures and can contribute to quality improvement in cancer surgery. A quality improvement program was implemented in the Province of Ontario that included the initiatives of: public reporting of postoperative mortality in pancreatic surgery, the dissemination of a set of standards for the delivery of such surgery, including recommendations for surgeon training, hospital resources and organization, and minimum volume requirements. Regionalization of complex services was encouraged by the provincial cancer agency and promoted through continuing education of practicing surgeons using formal presentations and informal mechanisms including Communities of Practice. These initiatives were associated with a reduction in the total number of hospitals doing pancreatic surgery, a shift in cases from low to higher volume hospitals and a decrease in both overall provincial mortality rates and mortality rates within each of the volume categories. It is our belief that the change in outcomes was the result of all of these initiatives together rather than any of them independently, and this coordinated systematic provincial quality improvement strategy is currently being applied in other disease sites.