Money matters: exploiting the data from outcomes research for quality improvement initiatives

Abstract

In recent years, there has been an increase in studies that have sought to identify predictors of treatment outcome and to examine the efficacy of surgical and non-surgical treatments. In addition to the scientific advancement associated with these studies per se, the hospitals and clinics where the studies are conducted may gain indirect financial benefit from participating in such projects as a result of the prestige derived from corporate social responsibility, a reputational lever used to reward such institutions. It is known that there is a positive association between corporate social performance and corporate financial performance. However, in addition to this, the research findings and the research staff can constitute resources from which the provider can reap a more direct benefit, by means of their contribution to quality control and improvement. Poor quality is costly. Patient satisfaction increases the chances that the patient will be a promoter of the provider to friends and colleagues. As such, involvement of the research staff in the improvement of the quality of care can ultimately result in economic revenue for the provider. The most advanced methodologies for continuous quality improvement (e.g., six-sigma) are data-driven and use statistical tools similar to those utilized in the traditional research setting. Given that these methods rely on the application of the scientific process to quality improvement, researchers have the adequate skills and mind-set to embrace them and thereby contribute effectively to the quality team. The aim of this article is to demonstrate by means of real-life examples how to utilize the findings of outcome studies for quality management in a manner similar to that used in the business community. It also aims to stimulate research groups to better understand that, by adopting a different perspective, their studies can be an additional resource for the healthcare provider. The change in perspective should stimulate researchers to go beyond the traditional studies examining predictors of treatment outcome and to see things instead in terms of the “bigger picture”, i.e., the improvement of the process outcome, the quality of the service.

Introduction

The growing emphasis on an evidence-based approach in the medical setting has led to a corresponding increase in the number and quality of studies examining the efficacy of surgical and non-surgical treatments. These studies are usually conducted in university hospitals and clinics that have an in-house research staff or that cooperate with academic research institutions. The studies are not commonly perceived by the care provider (hospitals and clinics) as being something from which they can benefit, from an economical point of view; in contrast, carrying out such research can sometimes be seen as a drain on resources. The research activities on treatment outcomes are merely seen as something that may indirectly benefit the institution in terms of prestige and corporate social responsibility. Nonetheless, a meta-analysis of 52 studies involving a total of 33,878 observations showed a positive association between corporate social performance and corporate financial performance with reputation as an important moderator (ρ = 0.73) [1]. The public endorsement for the social activity (i.e., research) represents a reputational lever that can economically reward companies for their social contributions. For this reason, the authors of the meta-analysis emphasized the need to be attentive to the perceptions of third parties (e.g., public groups and media) since the most important way to reap benefits is in the form of financial return from reputation. For health care providers the benefits may be twofold, since the social contribution in terms of research involves the health care setting itself. Therefore, this can be perceived externally as a social contribution that not only increases scientific knowledge per se but also serves to improve the quality of the service provided by the clinic or hospital. In general, the best companies are characterized by their high social responsibility and research activities [1].

However, the possibility of economic benefit from Corporate Social Responsibility activities is not a sufficiently persuasive argument for increasing investment in research—otherwise, all the public and private hospital and clinics would likely have their own research departments or research staff. Significantly, in all of this, one important factor is typically overlooked: research projects in the field of treatment outcomes and their predictors can be useful to the provider in a much more direct way in terms of quality improvement and the control of service performance. These studies examine in depth the aspects of a key process in the delivery of health care, and the use of the scientific method ensures the collection of reliable and valid data. The outcome data collected for research purposes can be used to improve and monitor the quality of the “service.” It is well known in the business community that improved quality means the optimization of costs and decreased wasted time [2, 3]. Hence, the findings of these studies and the skills and expertise of the research staff who conduct them are well suited to being integrated into or supporting the most advanced, data-driven corporate quality management systems. Improving outcomes and, in turn, patient satisfaction will increase the chance that the patient will be a promoter of the provider to friends and colleagues. Since the percentage of customers who are “promoters” [4] and customer satisfaction [5–7] are both associated with the growth rate of a company, these factors have important economical implications. The aim of this article, written by authors with different backgrounds (medical, research and business areas), is to show how to utilize the findings of outcome studies for quality management and economic purposes in the same manner as such data are used in the commercial environment. It is hoped that, by the end of the article, the reader will realize that research staff can be an additional resource for providers who wish to optimize their health care processes by introducing or implementing data-driven quality improvement systems.

Patient perspective in clinical studies

Studies on the outcome of treatment are an important part of medical research. While in the past attention was focused on outcomes from the clinician’s perspective, during the last decade there has been increasing recognition of the notion that the assessment of patient perceptions allows for a more complete evaluation of the patient’s health status [8–10]. For this reason, the use of patient-oriented questionnaires as a method of data collection has gained increasing popularity [11]. Accordingly, the efficacy of treatments is assessed not only on the basis of diagnostic imaging and clinician-based evaluations, but also in terms of perceived health improvements and quality of life. For example, determination of the minimal clinically important change for an outcome instrument is commonly defined according to the patient’s perception of what constitutes an important change [12]. The last 10–15 years have also seen an increase in studies that have sought to identify predictors of treatment outcome [11]. In these investigations, both objective clinical measures and patient-oriented information are used to understand more about the factors that predict good and bad outcomes. And again, the criteria commonly used to define a “bad” or “good” outcome are based on the patient’s perceptions [11]. Taken together, these studies can be used to identify the best treatment, to follow over time the effectiveness of new and old intervention strategies, and to predict the outcome of surgical and non-surgical interventions from baseline data. From a scientific point of view the outcome predictor studies can also be useful to design subsequent intervention studies, since the predictors identified are the most likely candidate independent variables that can be manipulated in order to influence outcome (=dependent variable). Indeed, although correlations do not imply causation, correlation is needed for causation to be proved [13, 14]. This all serves to confirm the value from a scientific and medical point of view of studies that assess the patient’s perception of outcome.

Customer perspective in quality improvement

To understand the important role of the customer perspective in quality improvement we should start with the definition of quality. To those not involved in quality improvement in a professional capacity, it might appear relatively simple to define quality; however, more than 2000 years after Plato invented this term, there is still great debate regarding the meaning of the word [15, 16]. The American Society for Quality (ASQ) defines quality as “a subjective term for which each person has his or her own definition [17]. In technical usage, quality can have two meanings: (1) the characteristics of a product or service that bear on its ability to satisfy stated or implies needs, (2) a product or services free of deficiencies” [2]. This definition means that the product or service should be able to consistently deliver potential customers’ needs, doing: “things right all the time” [2]. According to a User-based Approach, quality can be defined as “meeting or exceeding customer satisfaction” [15, 16, 18]. For some of the pioneers of the modern quality movement (Walter Stewhart, Edwards Deming and Joseph Juran), this is the most important definition of quality in the economics-setting [15, 16, 19]. An imbalance between customer expectation and what the service or product actually provides can cause customer defections, reduced quality, higher costs and wasted time [19]. Quality is a multidimensional construct, and the dimensions are specific for each category. In health care, experts in quality management and international health organizations (e.g., World Health Organization) are in the process of identifying the most relevant dimensions [20, 21]; once these have been identified, quality surveys should provide performance measures for each dimension. Meeting or where possible exceeding customer satisfaction is a prerequisite of being competitive on the market. For this reason, quality improvement initiatives are usually customer-oriented and measures of customer satisfaction involve the use of subjective instruments (questionnaires, interviews, etc.) to solicit the customer’s opinion.

The US Agency for Healthcare Research and Quality defines quality in health care as “doing the right thing, at the right time, in the right way, for the right person, and having the best possible results” [22]. This quality definition has similarities to that suggested by the ASQ. The quality measures in health care assess three components: structure (resources such as staff, equipment), process (therapeutic interventions, prescribing, interactions with patients) and outcomes (end results of health care such as mortality and patient satisfaction) [22–24]. The measures used to obtain the patients’ view can be classified into three categories: preferences, evaluations and reports. Wensing and Elwyn [25] defined preferences as patient ideas about what should occur in healthcare systems. Evaluations are patient “reactions” to their experience of health care and reports are objective observations (e.g., how long the patients had to spend in the waiting room). The choice of the type of measure depends on the aspect being assessed and the purpose of the evaluation (educational, certification, accreditation, quality control or quality improvement) [25]. One of the most widespread means of measuring processes and outcomes is the assessment of patient satisfaction (evaluation category). Outcome satisfaction is also one of the criteria for assessing the validity of process measures. Indeed, according to Chassin [23], a measure of process is valid when it is related to health outcomes (e.g., mortality, patient satisfaction, etc.). Hence, the responses to questions concerning satisfaction with treatment, typically used in treatment outcome studies, can also be seen as outcome measures in the quality control and improvement context.

Patients or customers?

Customers can be classified into two categories: internal and external customers. In health care, internal customers are directors, physicians, clinicians, nurses and employees. Patients are external customers [19]. The consideration of patients as customers is something that does not always sit well in health care. However, whether the stakeholder is referred to as a “patient”, “client” or “customer” depends only on the provider of the service and makes no substantial difference. Having a customer, a provider and a market, the methods and the philosophy used in industry to improve quality can be applied in the health care setting and they can certainly be useful to obtain benefits both in terms of quality improvement and economic revenue. The growing preoccupation with cost shifting and cost reduction is also increasing awareness of the fact that health care should be moving toward market-based strategies to improve performance [23, 26–29]. The notion of health care as a market is often viewed by health operators with distaste. This prejudice is mainly due to the lack of knowledge of market strategies, which are erroneously believed to be oriented only on cost reduction, with quality being readily sacrificed to contain costs. Instead, however, it is poor quality that costs money, and the improvement of quality that hence allows costs to be reduced [30]. It has recently been advocated that only with a shift to value-based competition, common in industry, will we be able to improve health and health-care value [29, 31], where value is a function of satisfaction of needs/use of resources [3].

Longitudinal studies of treatment outcome provide a mechanism by which patient satisfaction can be monitored over time; they hence provide the baseline data with which the effectiveness of quality improvement interventions, adopted by the provider, can be verified. Quality initiatives should be implemented within a stable process, i.e., in a “service” producing consistent outcomes. For example, in consistently monitoring patient satisfaction or the perceived outcome from the customer’s “perspective”, it is then possible to see whether individuals treated in a particular period of the year (or month or week) reported a lower satisfaction level. This can help in understanding what happened and how the problem can be addressed (the so call special causes) [2, 3, 32]. The periodic patient survey is essential to measure “the voice of the customer” in order to identify the opportunities for improvement and to set priorities among the identified opportunities [19]. After an improvement initiative has been implemented, the continued measurement of patient satisfaction allows evaluation of the extent of progress. Hence, the instruments used in outcome studies can serve to routinely or periodically monitor the process, to ensure that improvements are maintained and that the process does not regress to its initial level. An intervention that increases patient satisfaction also increases the likelihood that the patient will become a so-called “promoter”. In the business community one of the most common measures of customer loyalty is the “net promoter score” [4]. This measure is obtained asking the customer “On a scale of 0–10, how likely is that you would recommend us to your family, friends or colleagues?” The difference between the percentage of customers who give high responses (from 9 to 10; promoters) and those who give low responses (from 0 to 6; detractors) gives the net promoter score. The net promoter score is related to the company’s revenue growth [4, 33]. Kinney [34] have shown an example in healthcare using the net promoter score to calculate the potential lost revenue. This was done considering that a satisfied patient is likely to tell 3 customers (potential referral) and a dissatisfied patient is likely to tell 11 customers (lost referral) [34, 35]. Multiplying the revenue per patient service by the number of potential lost referrals it is possible to estimate the potential lost revenue [34]. It has been shown in industry and banking that customer satisfaction influences customer loyalty, which in turn affects profitability [36]. The influence on the net promoter score of patient satisfaction with treatment has yet to be quantified; however, it is likely that satisfaction will be an important determinant of whether a patient will become a promoter of the provider. The relationship between customer satisfaction and company growth has also been shown in various studies [5–7].

Quality management systems and six-sigma

Quality management has three main components: quality assurance, quality control and continuous quality improvement. The ASQ defines quality assurance as “planned and systematic activities implemented in a quality system so that quality requirements for a product or service will be fulfilled” [17]. Quality control is an integral component of quality assurance and consists of a system for verifying and maintaining a desired level of quality [22] and can include “observation techniques and activities used to fulfill requirements for quality” [17]. Continuous quality improvement is a systematic, organization-wide process focusing on linear, incremental improvement in the quality of services and the elimination of waste [17, 30, 37]. The development of quality systems started in the 20s with the work of Walter A. Shewhart of Bell Telephone Laboratories who developed a statistical control chart, which is considered the first method of quality assurance introduced in modern industry [2]. Since then several methods for quality management have been developed and are now available (see Box 1). Historically, the healthcare industry has mainly adopted quality assurance methods; only recently have continuous quality improvement approaches been embraced. As suggested by Varkey et al. [22], the best organizations combine quality assurance with continuous quality improvement.

Box 1.

List of available methods for quality management

9000 Series; ANSI/Mil-Std, ASQ/CQA; Benchmarking; Cause and Effect Analysis; Change Management; CMM-SEI; Control Charts; Customer Driven Processes; Decision Making; Design of Experiments; Fault Mode Evaluation Analysis; Knowledge Management; LEAN Thinking/Manufacturing; Malcolm Baldrige; Management by Fact; Mistake Proofing; Performance Measurement/Metrics; Poka-Yoke; Problem Solving; Process Capability; Process Management; Process Mapping; Process Re-engineering; Project Management; Root Cause Analysis; Set-Up Reduction; Six Sigma; Statistical Process Control; Totally Quality Management; Trend Analysis; Variation Measurement; Work Breakdown Structure; Work Flow Analysis

Six-sigma (6-σ) is a statistically-based quality improvement technique developed in the 80s by Mikel Harry an engineer of Motorola [2, 3, 38]. The method was subsequently popularized by AlliedSignal and General Electric, with the latter starting to use the 6-σ in an extensive and systematically way [38]. More recently General Electric has started to use the Lean-sigma, which includes both the Lean manufacturing (Toyota) and 6-σ philosophy. This customer-oriented process-focused approach to business improvement is now considered to be one of the most effective continuous quality improvement systems and it is used by the biggest company in the world to increase performance and decrease performance variation [2]. The basic unit for improvement is the “process”, which is defined as “a series of steps and activities that take inputs provided by suppliers, add value and provide outputs for their customers” or simply “a combination of inputs, actions and outputs” [32]. The 6-σ takes its name from the Greek letter indicating the standard deviation that is used to measure the process performance and variation. A measure of process performance indicates how well the process performs and it is assessed by comparing the actual versus the ideal process performance level or “specification limits” (e.g., customer requirements) [3, 39]. 6-σ refers to a process whose variability is such that 6 standard deviations fit between the mean performance value (center of the process) and the nearest specification limit (see Fig. 1). The means to achieving such quality is to reduce the rate of defects (or exceptions) so that the process variation is low in relation to the allowable tolerance (specification limits). A 6-σ process will produce 3.4 defects per million of opportunities (DPMO), i.e., doing it right 99.9997% of the time [38]. In Table 1, the σ level, the percent of process efficiency (“doing the right thing”) and the corresponding DPMO are presented. To take an example from the medical field, if we have 48 patients out of 300 who are not satisfied with their overall care, this means that our process works successfully 84% of the time; this corresponding to a σ level of about 2.5σ1 (160,000 DPMO). In reference to Table 1, it should be noted that the proportion of data contained within the range (defined by the multiples of σ) differs compared with the empirical rule for normal distribution curves commonly used by researchers for confidence intervals. For example, ±2σ corresponds to the inclusion of 69.2% data points (Table 1) rather than the typical 95.4%. To avoid confusion, it perhaps helps to explain that the process σ levels are commonly calculated using the so-called 1.5σ shift [39]. When Motorola developed the method they decided to adjust the mean by subtracting 1.5σ before determining the percentage out of specification. This was done in order to take into account variations (shift or drift) in the process mean over time due to uncontrollable factors. Conceptually this is the same as the difference between the standard error (estimate of the true population mean) and the standard deviation (an indicator or the variability of the data sample), or between the R² and adjusted R² [40]. While the use of a fixed adjustment is questionable from a statistical point of view (for example too conservative with bigger sample sizes), it has facilitated the development of a common language and standardized criteria and process metrics. A performance of 6σ represents more a model “desirable goal”, although in some sectors such as Airlines the performance is about 5σ (230 fatalities per million opportunities). In health care it was suggested that with the exclusion of deaths for anesthesia (close to 6σ or 5.4 deaths per million) the performance is generally poor or markedly lower than industry. In his article “Is health care ready for Six Sigma quality” Chassin [23] provided an example of the (inappropriate) use of antibiotics for treating colds with a defect rate of 21%. If the same performance were “translated” into the credit card industry, banks would deposit 36 millions checks in the wrong account, and airplane crashes would rise 100-fold.

Fig. 1.

Representation of the process performance according to the 6-σ definition (a 6σ process, b 2σ process, c 4σ process). DPMO defects per million of opportunities, LSL low specification limit, LCL low control limit (process center −3σ), UCL upper control limit (average process performance +3σ), USL upper specification limit, Process center average value of the process performance, Process spread process variability expressed as average process value ±3σ

Table 1.

Sigma (σ) levels with the corresponding defects per million of opportunities (DPMO) and process efficiency value (“doing the right thing”) calculated taking into account a σ shift of 1.5 to compensate for variations of the average process performance over time

σ levels	DPMO	Process efficiency (with 1.5σ shift) (%)
1	690,000	31
2	308,000	69.2
3	66,800	93.32
4	6,210	99.379
5	320	99.977
6	3.4	99.9997

For these reasons, the methodology has attracted increasing attention in the health care sector in the last 10 years and its use is growing [22, 23, 41–45]. Six-sigma has been successfully applied in various health care areas such as: radiology, to improve the through-put and thereby reduce the costs per radiology procedure; Supply Chain Management for surgical supplies, to reduce the inventory levels and improve supplier relationships; Emergency Departments, to address the problem of overcrowding, by reducing the time required to transfer patients from the Emergency Department to an inpatient bed and to achieve pain management; laboratory diagnostics, to decrease the analytical errors; infection control, to reduce post-operative and catheterization infections; and heart failure, to reduce the variations in outcomes, length of stay and treatment [45, 46]. The estimated hard cash saving of these improvement initiatives ranged from 60,000 to about 1 million US dollars per year. Overall, the 6-σ methodology has been advocated as a potentially effective quality method through which it should be possible to improve the health care system both in terms of the quality of the care provided and cost reduction [23, 45–48].

The 6-σ process improvement strategy involves five steps: define, measure, analyze, improve, control (DMAIC, Fig. 2a) [2, 3, 32, 38]. In the “Define” stage the project (teams, stakeholders, plans) and customer requirements are identified. In the “Measure” stage, data are collected for evaluating the current performance level of the process. This requires the identification of what needs to be measured and the development of a data collection plan to evaluate the robustness of the measurement systems. Subsequently the process performance is examined and the sigma level calculated. In the “Analyze” stage data are analyzed to accept or reject hypotheses about cause and effect, or to identify the variable most influencing the process. In this phase it should be also determined whether the process can be improved or instead needs to be redesigned through, for example, Design for Six-Sigma (DFSS) strategies. In the “Improve” stage the root causes of process performance are identified and a solution is generated. Finally, in the “Control” stage, the performance after the intervention is monitored using control charts, the documentation is formalized and the project is handed over to business (recognizing success). At this point the reader should have already recognized something familiar. Ignoring the specific tools, we can summarize the DMAIC process as follows: identification of the problem; identification of the appropriate dependent and independent variables (descriptive studies), determination of the research hypothesis, selection of the appropriate experimental design, collection of baseline data (pre-intervention), manipulation of the independent variable (intervention), examination of the effects of the intervention on the dependent variable (outcome). In other words, the 6-σ, one of the most advanced quality improvement methods, is actually the application of the scientific method to quality improvement. Of course, experts in 6-σ use specific instruments and techniques for their analyses, but the statistical techniques are actually very similar to those commonly used by researchers. Adopting a quality management perspective can modify our view of the studies that we are conducting at present or have done in the past. Sometimes projects start and end with the identification of predictors of outcomes. Embracing the 6-σ philosophy, for example, these descriptive studies would be seen as just a part of a bigger process (Analyze stage in the DMAIC roadmap) oriented towards improving some aspects of the quality of health care. The end of the project would be the examination of the effectiveness of the intervention selected for improving outcomes. Once the intervention has been proven to be effective, it can then be implemented in the routine process of the provider and the Control stage of the DMAIC will check the consistency of the results.

Fig. 2.

DMAIC process (modified from Chassin [68])

An ongoing example using Six-sigma methodology

In this section we present a very simplified example of a quality improvement initiative developed using the 6-σ approach and still ongoing (Fig. 2b).

Define

The process selected for this example was the surgical treatment of degenerative spinal disorders, with the aim of improving the quality of surgical treatment. Quality can be mathematically defined as the perceived performance ÷ expectation where performance is what the product or service can do for the customer from his or her perspective [2]. From this formula both numerator and denominator depend on the “customer” perception. This concept is familiar to researchers expert in treatment outcome studies since in these investigations the patient perspective is central. In our example, the selected measure of process outcome was the patient perception of the “process performance” using a question included in the COMI [11, 49]. This question asks the patient to rate the global effectiveness of the operation on a 5-point Likert-scale [operation: (1) helped a lot, (2) helped, (3) helped only a little, (4) did not help, (5) made things worse].2 Several factors can influence the outcome of a process and in quality management these are commonly classified and analyzed according to six categories (see the fishbone diagram, Fig. 3). Factors from each of these categories have previously been examined as predictors of outcome of spine surgery [e.g., 49]. Our quality intervention was centered on the so-called Mother Nature factor (patient intrinsic characteristics) and in particular on the importance of patient’s expectations in influencing their rating of the “process performance.” The selection of patient expectations as a potential predictor of outcome was based on both a literature review [e.g., 50–53], previous studies conducted in our department, and on the feedback of the spine surgeons regarding their own impression of why patients might be disappointed with their results following what might be viewed by the surgeons as a successful operation [54]; one important characteristic of the 6-σ is that it is based on an effective teamwork, where the opinions of all individuals involved in the process are taken into account.

Fig. 3.

Fishbone diagram: generic (a) and specific (b) to the Schulthess Clinic Spine Unit quality initiative presented in the text

Measure

According to the DMAIC framework, in this stage the performance of the process is measured. In the clinic where this initiative was carried out, the COMI is completed systematically by all patients in the Spine Center, before surgery (core outcome questions only) and at various stages after surgery (core outcome questions plus global outcome and satisfaction). The data analyzed were those collected from January to August 2008 relative to the results of spinal surgeries performed 12 months before. Although we had at our disposal 4 years of data, this more recent time frame was chosen to have data representative of a period just prior to the quality intervention. With this data the process capability and the corresponding sigma level were calculated. Considering as a defect (a “poor outcome”) a patient rating the process outcome from 3 (helped only a little) to 5 (made things worse) [49, 55], the current performance level was 2.24σ, which corresponds to 229,600 DPMO [56], i.e., doing the right thing about 77% of the times. As mentioned earlier, we decided to focus our attention on patient expectations, in order to try to improve the patients’ rating of success of the operation. Data regarding patients’ expectations had been collected by the SPINE research unit of the clinic in a previous prospective study [57] using a modified version of the North American Spine Society (NASS) Lumbar Spine Questionnaire. Briefly, before the operation, the patient was asked to rate their expectations using a 5-point Likert scale in relation to eight items: leg pain, back pain, walking capacity, independence in everyday activities, general physical capacity, ability to do sport, frequency and quality of social contacts, mental well-being. After surgery, the actual changes experienced in each of these dimensions and direct questions regarding the fulfillment of expectations were also asked. Hence, we had all the data to go on the subsequent step.

Analyze

In this stage the data are analyzed to suggest, support or reject theories relative to the cause of the defects. Such an analysis was done in the aforementioned study [57]. Firstly, it was found that the actual improvements after surgery were markedly less than those expected prior to the operation. Secondly, using hierarchical multiple regression analysis, not “expectations” per se but “expectations being fulfilled” was the most important predictor of global outcome. From this analysis the following points were raised:

1. the patients had generally overly optimistic expectations;

2. improvements in symptoms were important for a good outcome, but the fulfillment of (realistic) expectations appeared to be more important.

These findings are in line with those reported by McKinley et al. [50] who found that meeting or failing to meet patients’ expectations was an important independent predictor of patient satisfaction.

Improve

Once the root causes for the process performance have been hypothesized or identified, an improvement initiative should be generated, tested and implemented. From the two main findings of the previous stage, it was decided to focus during the pre-operative consultation on clearly explaining the improvements that could be expected after surgery, and then to document both the patient’s and the surgeon’s understanding of what was discussed using a questionnaire. The patient and surgeon were to complete the questionnaire independently after the consultation. This intervention is currently in an initial pilot phase with one surgeon. The ultimate aim—and especially if it should transpire that there are still large discrepancies in what the patient understands and what the surgeon thinks he/she has explained—would be for the surgeon to complete the form together with the patient, as a sort of agreement between them concerning the likely benefits of the surgical treatment. The “management” of patient expectations has also been suggested by McKinley et al. [50] as a solution to improve patient satisfaction in out-of-hours care. The necessity for more active involvement of the patient in health care (patient centered care) has also been advocated by some [58]. However, it has also been highlighted that, whilst this approach is timely, it may be impractical [59]. Indeed, Dunn [59] indicated that patient-centeredness “is not a cheap option, in terms either of staffing time and resources.” However, this solution was considered acceptable by the medical staff of the SPINE unit and therefore used in this quality initiative. In order to contain the cost of this intervention, the physician changed the structure of the consultation without increasing the total time of the visit. In another unit of our clinic (Hip unit) that is introducing a similar quality improvement initiative, the solution selected is to create and give the patient a booklet specifically designed to “manage” expectations. This latter solution is less “intrusive” since it does not require surgeons to change the content or structure of the consultation. Both solutions are, however, cheap options. Our examples end here, since these initiatives are still ongoing and the solutions proposed are still in the testing phase. Once the solution has been implemented in the process, data on perceived performance will once again be analyzed such that the results can be controlled on a quantitative basis (CONTROL stage).

Use of treatment outcome data for process control

In this last section we showed an example of how to use data collected within the framework of treatment outcome studies, for quality control purposes. We have already mentioned that a quality initiative must be implemented within a stable process. But what is a “stable process” and how can we monitor or “control” whether it is stable? In the following paragraph we will try to address these issues. Any process contains two sources of variation: “common (or random) causes” and “special causes” [60]. The common causes are the sum of a multitude of effects determined by the interaction of random factors such as (in our case) surgical equipment, surgeon technical performance, patient intrinsic characteristics, etc. Common causes affect every outcome of the process and everyone working in the process [61]. When the only sources of variation come from common causes, the process is said to be “stable”, “in statistical control” or simply “in-control.” In statistical terms, this means that the variation of the process outcome (e.g., patient satisfaction, or patient perceived improvement after a treatment) is randomly distributed around the average performance. Since the observed variation of the process outcome tends to follow a distribution function, it will be predictable. Conversely, special causes refer to sources of variation that are not an inherent part of the process but are due to unusual circumstances [60, 61]. This determines excessive and unpredictable variations in the process performance. When special causes are present the process is said to be “unstable”, “out of statistical control”, or “out-of-control” and not predictable. The management of common causes of variation requires the improvement of the process (actions on the system), while for special causes these unusual sources of variations must be located and removed (local actions) [61]. When a process is stable, its “capability” can be calculated, where capability is the extent of process variation that comes from common causes only and is usually expressed in relation to specification limits (target).

How can a process be evaluated for common and special causes of variation? One of the most powerful tools used to examine the processes is the so-called “control-chart.” The control chart is a graph utilized to examine how a process changes over time [60, 62]. The typical control chart for normally distributed interval data consists of a central line for the average process performance value, an upper line for the upper control limit and a lower line for the lower control limit. These control limits are calculated as three times the variability of the data (i.e., ±3σ) calculated from historical data. By comparing current data to these lines, it is possible to draw conclusions about whether the process variation is consistent (in control) or is unpredictable (out of control) [60, 62]. Furthermore, it is possible to draw on the chart warning lines that usually correspond to 2σ (Fig. 4, left panel). From this graph it is possible to detect so-called “out-of-control” signals indicating that a special cause occurred, as follows: (1) a single point outside the control limits; (2) two out of three successive points on the same side of the centerline and farther than 2σ from it; (3) four out of five successive points on the same side of the centerline and farther than 1σ from it; (4) a run of eight in a row on the same side of the centerline; (5) obvious consistent or persistent patterns that suggest something unusual about the data and the process [62]. In Fig. 4 (right panel) we have calculated a special type of control chart (p-chart for attribute) using historical data (2004–2008) relative to the perceived performance as measured 3 months after spinal surgery using the Global Outcome question of the COMI. As mentioned earlier, we considered a patient rating from 3 (helped only a little) to 5 (made things worse) to be a process “defective unit” [55, 63]. No signs of an “out-of-control” process were detected, indicating that the variations in the process were due to common causes only. Collecting the perceived performance data on a sample of patients or systematically on all patients can allow the process to be evaluated on a monthly basis. If an out-of-control signal is detected, actions can be adopted to understand the cause of variation so that it can be removed. Analysis of the control charts also allows the identification of trends in the process over time. Furthermore, having a stable process allows the implementation of quality improvement initiatives to decrease the variation due to common causes, as in the example using 6-σ methodology described in the previous section.

Fig. 4.

The left panel represents a generic control chart, and the right panel shows the control chart (p-chart for attribute) indicating the proportion (p) of defective units [patients rating from 3 (helped only a little) to 5 (made things worse) in the global outcome question over 48 months]

Contribution of the research staff to the quality improvement systems

There is general consensus that an improvement in the quality of a service can be attained with the active cooperation of all the operators [23, 29, 45, 64, 65]. The need for a value-based competition market to improve quality will increase the necessity to incorporate industry-derived methods in health care. We have focused this article on the 6-σ because, among the several quality improvement systems, this method drives process improvements through statistical measurements and analyses. Given that 6-σ is data-driven and relies on the application of the scientific process to quality improvement, researchers have the necessary skills and mind-set to embrace it and to effectively contribute to the quality team. Furthermore, it appears to be one of the most effective systems and it can easily be integrated into other methodologies such as the more traditional Totally Quality Management [65]. Depending on the problem, other methods that have not been elaborated on in the present article can also be used to good effect (see Box 1). However, irrespective of the method used, the change in perspective should stimulate researchers to go beyond the traditional studies examining the predictors of treatment and focus also on the final goal, the improvement of the process outcome, i.e., the quality of the service. A more economically-oriented view should also help the researcher to select the specific improvement solutions to be implemented, taking into account the cost:benefit ratio. This has important implications since a cheap but effective solution can be introduced more easily and quickly in routine practice, especially in non-government institutions. Improvement of the quality results in an increase in the value of the service, with more satisfied patients; these will likely become promoters of the provider, hence giving an economic return. If adequately trained, the research staff are therefore well-suited for being part of the most advanced quality systems. Furthermore, researchers working in the clinical setting are expert in the use of patient-oriented tools and can provide support in validating the questionnaires to be used in quality management and in addressing their shortcomings. For example, while in the clinical setting interest in how to use parametric tests for ordinal data (e.g., using Rasch analysis [66, 67]) is increasing, in the business community this has not yet been addressed.

This paper offers an example of how the outcome studies of the clinical researcher can contribute to quality improvement systems. It aims to stimulate research groups to better understand that, by adopting a slightly different perspective, their studies can be an additional resource for the healthcare provider. Given the current quest for well established improvement methods in health care [23, 27, 45], the research staff represent an ideal door through which advanced quality improvement methodologies such as the 6-σ can enter the health care environment.