Measuring Quality of Surgical Care: Is It Attainable?

Healthcare issues are again a hot topic in this year's elections. U.S. healthcare spending is among the highest in the world, averaging $7,026 per person, or $2.1 trillion in 2006 and is growing at a rate of over 6.7% per year.(1-3) (Figure 1) The U.S. now spends approximately 16% of her Gross Domestic Product (GDP) on healthcare.(3) Despite this ever-increasing spending, it has been consistently shown that healthcare quality indices in the U.S. lag behind that of other industrialized countries.(2, 4) Furthermore, a growing segment of the U.S. population cannot afford healthcare; in 2007 nearly 15% of Americans did not have healthcare insurance.(5) This unsustainable increase in healthcare spending, coupled with a looming recession, has the potential to have great repercussion on the well-being of the nation. The unrestrained consumption of healthcare will eventually hamper the provision of other essential services, such as education, infrastructure, and defense.

Figure 1.

National Health Expenditures by year, 1960-2006 (billions of dollars)

Source: Centers for Medicare & Medicaid Services, Office of the Actuary: Data from the National Health Statistics Group.

Many nationwide experiments have been performed in an effort to decrease this growth. Most notably, the health maintenance organization (HMO) movement in the 1980s temporarily slowed the growth of health care through limiting access. This dip in the growth of health care consumption was made possible by cuts of services to the consumers, while managers of these healthcare plans raked up huge profits at the expense of the American people.(6) Capitation is an even more controversial way that has been used to try to contain costs.(7) At its heart, capitation is an agreement entered into by health care providers and payers to preset which services are covered and the agreed upon reimbursement rates for those services. The most notable capitation experiment has been the Oregon Health Plan, which began operation in 1994.(8) The plan expanded Medicaid benefits to more residents, but did so by reducing covered services. This was done by means of a prioritized list of more than 700 diagnoses and treatments. The Oregon State Legislature decided on a cut-off point on the list in which services below the cut-off point would not be covered. The plan saw early success in reducing uncompensated hospital visits and increasing the number of residents with health insurance, but it was hypothesized that this success could not be sustained.(9) As predicted, the program was not able to withstand inflation of medical costs and cuts to the plan, and eventually resulted in the closing of the program to new enrollees. In March 2008, enrollment was begun again with a lottery to fill 3,000 available enrollee spots, and more than 80,000 people applied.(10) Despite the short-coming of the Oregon Health Plan, Donald Berwick, MD, an authority on the quality of health care, theorizes that capitation can be beneficial to the American healthcare system, provided that reimbursement and covered services decisions are made in a collaborative way by encouraging the examination of provider decision-making processes.(11)

Recent Initiatives to Assess Quality

The outcomes movement, initiated in the 1980s, was proclaimed to be a potential solution to improve health care quality while restraining growth of expenditures.(12) While the outcomes movement has provided some insight into health disparities of the current healthcare system,(13-15) it has been unable to quell the increase in healthcare spending.(3) Furthermore, many providers felt that the data collection effort required to measure provider performance was an intrusion and threatened their independence.(14) Therefore, it is not surprising that the current focus on quality of care is met with great skepticism among physicians in general and surgeons in particular, whose decision-making ability has been challenged at all levels of patient care. Many physicians feel that past “quality improvement” initiatives focused on finding fault and served only to enforced “punitive, sometimes humiliating sanctions” and, in general, made “physicians” lives difficult.”(16) Because of this, they see enthusiasm for quality of care as a façade for even more incursion into the patient-physician relationship and as a thinly-veiled attempt by government, industry, and managed care companies to introduce additional bureaucracy in the already taxing practice of medicine.(16) Hence, in 2003, when the Center for Medicare and Medicaid Services (CMS) introduced a pay-for-performance initiative, this was met by skepticism, anger, and, predictably, disinterest among many specialty societies.(17) These feeling were particularly prevalent in certain surgical societies who feel that the metrics used for measuring quality in surgical care are not accurate, and the payment for participation in these quality of care initiatives are meager at best to justify support.(18)

Surgeons’ Role in the Quality Movement

However, surgeons cannot afford to stand on the sideline as passive observers of these monumental changes in the direction of American healthcare. As David Blumenthal, MD observed in the first of a six paper series on quality of healthcare published by the New England Journal of Medicine, the medical profession's legal and economic privilege to practice medicine is granted by the public.(19) When in the past, most decisions made by physicians went unchallenged, today there is a swelling sentiment among the public that American medicine is failing. Consequently, the pubic is demanding more accountability and transparency regarding the quality of care that they are receiving. These feeling will only heighten as subsidies for services become scarce, and the public assumes more responsibility for paying for their healthcare. Governmental agencies and insurance industry have all publicly endorsed support for accountability of quality of care and have taken a leadership role in crafting some of the quality initiatives in an effort to appease public outcry and to maintain control of their investment in healthcare. While the feelings of distrust and disinterest among physician and surgeons are understandable, it is the medical professionals who have an intimate knowledge regarding what constitutes high quality care will be the biggest losers if they do not actively participate in these national discussions.

What is Quality of Healthcare?

When discussing the concepts of quality in health care, the conversation will inevitably turn to the scholarly works of Avedis Donabedian, MD, a prominent public health figure from the University of Michigan School of Public Health. Dr. Donabedian conceptualized quality of healthcare into 3 components: 1) Structure, 2) Process and 3) Outcome.(20, 21) (Figure 2) Structure relates to the physical facility where healthcare is provided, for example, the number of beds in a hospital or the ratio of nurses to patients in a particular unit. Process indicates the conduct of healthcare delivery, for example, whether pre-operative antibiotics are given before elective surgery or whether an electrocardiogram is obtained for patients over 50 years of age. Outcome assessments are, quite simply, the status of the patients following care.

Figure 2.

Donabedian's Three Components of Healthcare Quality

Adapted from Donabedian (21)

In the past, researchers have been focused on outcomes, particularly in the surgical discipline.(22) Outcomes measure the end results of patients’ treatment and, unfortunately, may be quite elusive for certain procedures or conditions. For example, outcomes after breast reduction surgery cannot be measured by standard physical tests but rely strongly on patient-reported assessment of symptom alleviation. For certain procedures, such as joint replacement surgery, certain outcomes, such as joint loosening or wear, may take many years to develop, making it difficult for physicians to understand this procedure's effectiveness and complication rate.

Today, health services researchers increasingly have turned to structure and process measure as a proxy of quality of care because most structure and process indices can be measured rather easily and precisely. Both structure and process are thought to be adequate quality measures. However, some have observed that despite poor structure and process performance in certain facilities, outcomes do not suffer in many patients. For example, not every patient who does not receive pre-operative antibiotics will develop a wound infection. But a patient who does not have an adverse event due to luck alone is not a justification for the substandard quality of care that he/she receives. Therefore, some researchers may still prefer outcome measures as a more pertinent quality metric. But again, outcomes can be difficult to measure, often taking a long time to assess, and the evaluation of outcomes depends on whether one's perspective is patient-rated or physician-centered.

Current National Quality Initiatives

Although the metrics used to measure quality are currently imprecise, there is great hope that eventually better quality measures can be developed. With the input of the medical profession, these quality measures can continue to be refined so that their reliability and validity can be tested in a scientific fashion. The American Society of Plastic Surgeons (ASPS) has taken a proactive approach to be involved in these national discussions along with the American College of Surgeons. Although the process of engagement is laborious, the crafting of reasonable quality metrics for surgeons may be based on an understanding of the uniqueness of the surgical specialties. It is important for surgeons, in particular plastic surgeons, to participate in these discussions so that we can educate others in the appropriate use of quality measures in surgical care and be a leading advocate for our patients to obtain the highest quality care.

The Agency for Healthcare Research and Quality (AHRQ) has undertaken the mission to support the improvement of health outcomes by strengthening the quality measurements and to reduce unnecessary healthcare expenditures.(23) The main approach taken by AHRQ is the issuing of evidence-based clinical practice guidelines. These guidelines are developed and submitted by various professional societies and organizations, including ASPS. However, these guidelines can become outdated if not regularly rechecked for validity,(24) a daunting task when one considers there are currently over 2000 guidelines in the AHRQ's National Guideline Clearinghouse.(25)

Specific to Surgery, the Veterans Health Administration in the Department of Veterans Affairs (VA) established the VA National Surgical Quality Improvement Program (NSQIP) in 1994.(26) NSQIP supplies almost constant feedback to providers on major operations in 9 surgical specialties.(26) This feedback, which includes both rankings and outcomes, allows individual VA hospitals to monitor their own quality. In 1998, NSQIP began its Private Sector Initiative, which only provides feedback to general and vascular surgery, at participating academic medical centers and community hospitals.(27) Part of the NSQIP's success lies in its attention to case risk-adjustment.(26) There is a general consensus that adjusting any outcome-based quality score for case-mix is important to avoid “punishing” hospitals and surgeons who take on more complicated cases or patients with multiple comorbidities, which will naturally have a higher risk of poor outcomes.(28) However, the main outcomes measured by NSQIP are 30-day morbidity and mortality, which are only relevant for certain high-risk procedures.

The pay-for-performance initiative, as championed by CMS, is a quality initiative to reward surgeons and physicians who adhere to quality metrics as determined by CMS. As expected, many complain that these measures do not accurately indicate quality of care.(29) In addition, the incentives for adhering to the quality checklist are quite small, accounting for only 1% of Medicare/Medicaid reimbursement. The program also adds additional responsibilities for the participants. For example, participating institutions will need to hire staff to monitor these metrics, often costing more than the potential incentives provided. Many quality workgroups have organized an effort to steer these national dialogues of quality measures despite the so-called “moving target” of these quality indices. The ASPS has been actively involved in these organizations to lend a voice for Surgery to serve as a constructive force rather than a sweeping resignation that these quality efforts have no place in surgical practices.

A New Approach to Healthcare Quality

In the interest of expediting quality of care initiatives, some health policy-makers have looked upon the total quality initiative based on the Toyota model.(30) The main tenant of the Toyota model is Toyota's total devotion to the pursuit of perfection in their products. When Toyota launched the Corolla model in the 1968, it was derided as an uncompetitive product in the lucrative US market. At that time, gas in America was relatively inexpensive. The “Big Three” US automakers were making large, powerful, gas-inefficient models that had remained unchallenged for a number of decades.(30) When the majority of households had only one vehicle, the Corolla was deemed too small for the average American family. In addition, its engine, while gas-efficient, lacked the power American consumers were used to. The vehicle's small size did nothing to improve its image. Compact cars were considered “entry-level” vehicles in the US market, a low price with even lower quality.(31) Those that did take a chance on the Corolla found that it did not fall victim to the American notion of the compact car. Word spread about the high quality of Toyota's products and sale skyrocketed.

Toyota was able to achieve this feat by adopting Edward Deming's concept of quality, also known as Lean Manufacturing. The implementation of a lean process includes examining the execution of an activity and locating areas of inconsistency and eliminating unnecessary steps. (Figure 3) (32) Ideally, it should be an iterative process, with continual examination and improvement. For example, Kim et al had success applying the lean production method to combat delays in the placement of peripherally inserted central catheters (PICC) for intravenous medication administration at the University of Michigan.(32) They examined the process for placement of PICCs and identified a time-consuming step that could be eliminated. By constantly reexamining the process, they were able to identify another area where standardization would make the process go smoother. The ultimate result was a 36% decrease in the average time between PICC request and line placement.(32) Although remarkable strides can be made using the Toyota model, one should not be misled that the automotive industry is exactly similar to the health care industry. Whereas a company can have several products and devote its entire production force to focus on its inventory, the medical industry has to tackle many diseases that are constantly changing, not mentioning the unpredictability of the social and economic conditions facing each patient. Egalitarian treatment of all patients as equal cannot be fully applied to the medical setting because patient factors such as ethnicity, gender, socioeconomic status, and a whole host of other intangible factors can affect patient outcomes. However, when initiating a quality of care program, the Toyota quality model provides some justification for those examining quality of care by choosing the so-called “low-lying fruit” of relatively easily measured process metrics.(32)

Figure 3.

The Lean Manufacturing Cycle of Quality Improvement

Adapted from Kim et al (32)

The Case for Process Measures

Facilities and personnel in a health care system usually do not change very much or change at a very rapid rate. And outcomes measurement is fraught with the aforementioned difficulties and cost. It is not surprising that process measures have naturally become an attractive target measure of quality.

Process can be measured either implicitly or explicitly.(33) Implicit measures have no prior standards established, but rather the classification relies on the opinion of the reviewing physician. For example, wound care centers have different outcomes and yet the quality of care provided can be classified as excellent, average or poor based on the subjective chart reviews by assigned auditors. More current process measures include the use of explicit process criteria in which checklists are proposed, such as ensuring that the patient received pre-operative antibiotics for certain surgical procedures or that a patient suspected to have carpal tunnel syndrome underwent a confirmatory electrodiagnostic study prior to carpal tunnel surgery. Explicit process measures are much more promising by providing vigorous scientific data to justify these assessment criteria.

Traditionally, measures such as those used by the Healthcare Effectiveness Data and Information Set (HEDIS) have been based upon administrative data.(34) But these measures are often imprecise(35) and have been criticized for poor data quality and poor translation to actual quality of practice.(33) Newer measures are being developed by CMS and purportedly are better, more sophisticated, aggregate composite quality measures. Unfortunately, case-mix adjustment can be quite difficult to perform and still remains a challenge. Practitioners are concerned that certain centers will be penalized because they tend to treat the sickest patients and most complicated cases. Another challenge in process measures for surgery care is that although a checklist can be structured to measure the process leading patients from clinic visit to the operating room, measuring the process of care based on the skill performing the surgical procedures is still at the rudimentary stages. There is no question that a surgeon's skill can play an important role in patient outcomes. It is quite difficult, however, to quantify the quality of conducting a surgical procedure and to link this to the overall quality measures.

Much attention should be paid to the development of proper process measures in Surgery. Poor, ill-conceived measures of quality can be extremely harmful to the overall health care delivery system. Injudicious application of measures that have not been scientifically proven and tested may erroneously label a particular institution or practice as a poor quality center and are thereby penalizing this institution unfairly. Therefore, it is most important that all specialties be adequately consulted when designing these quality measures so that all voices are heard.

After the measures are constructed, they should be field-tested for reliability, validity, and responsiveness. For example, for a measure to be reliable, an institution must demonstrate similar quality score in repeat testing when is no changes in the practice patterns. For it to be valid, discriminant validity must be shown; a low score center will have poor outcomes whereas a high scoring center will have better outcomes. Responsiveness is also important. A low scoring center, after instituting appropriate quality improvement processes, will have better outcomes on follow-up testing. Khan et al's 1990 examination of hospital payment processes provided a good example of the latter two attributes.(36) They were able to show that hospitals that scored lower on the process measure had poorer outcomes. In addition, they demonstrated that hospitals that had previously scored low were able to raise their scores after proper institution of the process.(36) Scientific approaches to tie process measures with eventual outcomes are the next step in the laborious process of validating these process indices.

Practically thinking, in Plastic Surgery, linking processes of care to outcomes can be very challenging. In certain specialties, such as cardiac surgery, quality of care is tied to mortality. For coronary artery bypass grafting, quality of care is generally measured by 30-day mortality, which is very easy to measure; at the end of 30-days, the patient is either dead or alive. Mortality, however, is not a measure that is usually applicable to Plastic Surgery. Therefore, greater effort should be placed on patient-rated outcomes measures. After all, many Plastic Surgery interventions improve health-related quality-of-life and outcome questionnaires for plastic surgery, such as the Breast Evaluation Questionnaire,(37) and for hand surgery, such as the Michigan Hand Outcomes Questionnaire(MHQ),(37-40) are important assessments of outcomes after implementing the process measures.

Concluding Thoughts

American medicine is embarking on an exciting experiment to devise quality measures to improve the overall health of the American people. The surgical societies, including the ASPS, have a duty and an interest, to participate in these national initiatives. Voices of surgeons must be heard; surgical specialties must demand a scientific approach in instituting these quality measures. These efforts must be constructive by avoiding the traditional approach of penalizing low performing centers and not providing the sufficient guidance to improve the performance of these centers to acceptable levels. Constructive participation by ASPS and her members in these quality initiatives places our society in a unique position among Surgery specialties to spearhead the quality of care movement and to fulfill our ultimate desire to place our patients’ interests first.