Quality Improvement in Postoperative Pain Management

Abstract

Introduction

Acute postoperative pain management is still far from satisfactory despite the availability of high-quality guidelines and advanced pain management techniques.

Methods

An outcome-oriented project called QUIPS (Quality Improvement in Postoperative Pain Management) was developed, consisting of standardized data acquisition and an analysis of quality and process indicators.

Results

After validation of the questionnaire, a total of 12 389 data sets were collected from 30 departments in six participating hospitals. Improved outcomes (reduction in pain intensity) were observed in four of the six hospitals. The most painful operations, in the patients’ judgment, were traumatological and orthopedic procedures, as well as laparoscopic appendectomy. Traditional process indicators, such as routine pain documentation, were only poorly correlated with outcomes.

Discussion

QUIPS shows that outcomes in postoperative pain management can be measured and compared in routine clinical practice. This may lead to improved care. QUIPS reveals which operations are the most painful. Quality improvement initiatives should use as few resources as possible, measure the quality of the outcomes, and provide rapid feedback. Structural and process parameters should be continuously reevaluated to determine their suitability as indicators of quality.

Acute postoperative sequelae, such as pain, nausea, and drowsiness, affect not only patients’ general well-being, but can also increase perioperative morbidity, the length of hospital stays, and the risk of developing chronic pain. Managing pain and other perioperative symptoms appropriately can minimize these consequences and is, above all, an ethical imperative. Yet despite the availability of high-quality guidelines and advanced pain management techniques, acute postoperative pain management is still far from satisfactory (1, 2).

One of the reasons for this discrepancy is the lack of suitable data on outcomes: it is difficult for hospitals to evaluate and compare their services, because standardized outcome assessments are the exception, rather than the rule. As a result, quality management in many segments of the healthcare system does not extend beyond the evaluation of structure and process. From the perspective of the patient, however, quality of care can be measured only in terms of outcomes (4). Indeed, the important role that measures of outcome play in quality management has been underscored in clinical studies (5, 6), methodological recommendations (4, 7), and the current guidelines of the Association of Scientific Medical Societies in Germany (Arbeitsgemeinschaft Wissenschaftlicher Medizinischer Fachgesellschaften; AWMF) (8).

To address this deficit, an outcome-oriented project called QUIPS was developed to improve quality in postoperative pain management based on the authors’ many years of experience in this field (9). The project focuses on selecting, analyzing, and benchmarking outcomes in postoperative pain management. Initiatives for improving pain management must be judged according to their impact on the quality of treatment, to the methodological quality of the measures they use, and to their relevance to everyday practice.

This paper will explore how outcomes in participating hospitals changed during the QUIPS project and whether interventions that took place during the study period were associated with these changes. It will also describe differences in postoperative pain intensity according to surgical discipline and the type of surgery performed, and will analyze the relationship between traditional process indicators and the outcome-oriented indicators explored in this project. Finally, the paper will discuss how best to incorporate the QUIPS project into an overall concept for improving postoperative pain management.

Methods

The QUIPS project

This project was designed for hospitals that provide surgical care in various disciplines. Data on outcomes and clinical measures were collected during patients’ first day after surgery in the respective departments using a questionnaire specially developed for this project. The questionnaire was divided into sections dealing with pain intensity, functional impairment, the side effects of pain treatment, and global assessment by the patient (table 1). Data on selected clinical and process measures were also collected, including the type of surgery and anesthesia, choice of pain treatment, and pain documentation. The questionnaire was administered by individuals who were not involved in the care of the patients in question (i.e. a documentation assistant, student, or nurse from a different department), and each patient was chosen in a randomized fashion.

Table 1. Overview of outcome measures on the questionnaire.

Outcome measure	Scale
Pain on ambulation	NRS 0–10*1
Maximum pain intensity since surgery	NRS 0–10
Minimum pain intensity since surgery	NRS 0–10
Is pain interfering with your mobility or movement?	Yes/No
Are you experiencing pain when you cough or breathe deeply?	Yes/No
Were you woken up by pain last night?	Yes/No
Is pain interfering with your mood?	Yes/No
Have you felt very tired since your surgery?	Yes/No
Have you felt nauseous since your surgery?	Yes/No
Have you vomited since your surgery?	Yes/No
Would you have liked to have received more pain medication?	Yes/No
How satisfied are you with your pain treatment since surgery?	NRS 0–15*2

^*1 0 = no pain, 10 = most intense pain imaginable

^*2 0 = very unsatisfied, 15 = very satisfied

To ensure that data were collected under standardized conditions, the authors developed written guidelines and provided training to study personnel. Data were entered using an input mask and subsequently transferred to a central computer (i.e. a so-called benchmark server), which allowed participants real-time access to outcome data by means of a configurable, web-based feedback function (figure 1). These data were presented in such a way that participants could compare the outcomes in their own department to those in the departments of other participating hospitals. This allowed for external benchmarking according to surgical discipline. Aided by this outcome feedback, participants were able to explore different ways of exchanging their experiences, thus facilitating the identification of best clinical practice. For this purpose, eight benchmark meetings were held, during which outcomes were analyzed, treatment concepts were exchanged, and interventions were planned. In the member area of the project website, the treatment concepts of hospitals with above-average performance were presented anonymously as a way to encourage learning from best practice; information on potential pitfalls was also provided.

Figure 1.

External benchmarking: the results from one’s own clinic are marked for easy identification; the results from other clinics are anonymous. Different outcome measures, surgical disciplines, and time spans can be selected. Analysis results are updated continuously; significant differences are marked. SD = standard deviation; stat. sig. = statistical significance

Data collection

At the beginning of the project, 300 patients at each hospital were surveyed using the outcome measure "Maximum pain intensity since surgery" to determine the baseline status of pain management. After a preparatory phase, continuous data collection was initiated and a total of 12 389 data sets were collected over a period of two years.

Data analysis

Validation: Estimates of internal consistency reliability for measures with numerical rating scales (i.e. pain intensity) were calculated using Cronbach’s alpha and for dichotomous items (i.e. functional impairment) using the Kuder-Richardson Formula 20. To assess differential validity, nonparametric methods (i.e. the Mann-Whitney U test or chi-square test) were used to compare pain intensity and functional impairment for two separate, frequently performed surgical procedures.

Changes in the quality of pain treatment: To assess changes in the quality of pain treatment, the results of the baseline survey were compared to data from the first and last quarters of the benchmarking period. The question on "Maximum pain intensity since surgery" served as the measure of outcome. This measure was also evaluated by the authors according to surgical discipline. For the analysis of especially painful operations, all surgical procedures that had been performed at least 25 times and in no fewer than half of the hospitals were classified according to maximum pain intensity. As an example of an intervention that took place during the observation period, the authors chose the withdrawal of rofecoxib from the market in October 2004, because this involved a clinically relevant question and sufficient data were available. The Mann-Whitney U test was used for statistical analysis. Because the abovementioned analyses involve observational data and are of a descriptive nature, the authors did not set significance levels or perform multiple testing. As a result, the given P values are to be understood as exploratory.

Correlation between process indicators and outcome measures: For purposes of estimation, the authors investigated whether routine documentation of pain intensity correlated with the outcome measures in the questionnaire. This process indicator was chosen because it is described as an obligatory measure of quality in almost all guidelines and recommendations, as well as in the current S3 AWMF guidelines (8). In addition, the authors examined whether the measure "Desire for pain medication," which mirrors the process "Availability of pain medication" from the perspective of the patient, correlated with other outcome measures. To analyze correlation with the outcome measures, the authors calculated Spearman’s rho for ordinal data and the phi coefficient for dichotomous data.

Results

During the phase funded by the German Federal Ministry of Health between 1 October 2003 and 30 September 2006, 30 departments in six hospitals took part in the project. The hospitals were of various sizes (i.e. two university hospitals and four medium-sized hospitals) and provided surgical care in different disciplines. After a preparation and training phase, and following a survey of the baseline situation, continuous data collection and feedback were initiated in the summer of 2004. Over a period of two years, a total of 12 389 data sets were collected; the mean number of data sets per hospital was 2064 ± 393. Altogether, 48.2% (n = 5970) of the patients were women, and the median age in the overall population was between 51 and 60 years.

Validation of the survey instrument: For both measures of pain intensity (i.e. pain on ambulation; maximum pain intensity) a Cronbach’s alpha of 0.84 was calculated for internal consistency; for the dichotomous items, a Kuder-Richardson Formula 20 coefficient of 0.54 was obtained. To assess differential validity, the authors compared two procedures with each other: cholecystectomy (German surgical procedure code [OPS] 5–511.11, n = 188) and hip replacement (OPS 5–820.00, n = 169). There was a significant difference between the two procedures both in terms of pain intensity and functional impairment (maximum pain intensity: P = 0.001; pain on ambulation: P = 0.007; pain on movement: P = 0.004; pain on breathing: P < 0.001). Cholecystectomy was generally rated as being more painful (maximum pain intensity: 4.1 versus 3.2).

Changes in the quality of treatment: Measured according to the outcome "Maximum pain intensity since surgery," five of the six participating hospitals showed an improvement, and one hospital showed worsening, in the quality of treatment six months after benchmarking was initiated. Four hospitals were able to maintain this improvement throughout the remaining project. However, by the end of the fourth quarter, one hospital showed a slight worsening, and another hospital showed a more substantial worsening, in pain management compared to the baseline survey (figure 2). All changes had a P value less than 0.05 compared to the baseline measurement (e-tables 1, 2).

Figure 2.

Changes in the measure "Maximum pain intensity since surgery" for all six hospitals during the phase funded by the German Federal Ministry of Health

e-Table 1. Population size, mean, standard deviation for "Maximum pain intensity since surgery" over time for the six participating hospitals.

	Baseline survey			2nd quarter benchmark			Last quarter benchmark
	N	Mean	SD	N	Mean	SD	N	Mean	SD
Hospital 1	163	4.41	2.88	351	2.50	2.40	210	3.36	2.62
Hospital 2	226	5.11	2.57	213	5.95	2.39	246	5.45	2.59
Hospital 3	153	5.42	2.62	100	4.61	2.47	290	5.01	2.75
Hospital 4	193	4.34	2.44	169	3.67	2.26	243	3.53	2.20
Hospital 5	334	4.40	2.64	209	3.69	2.26	137	5.36	2.34
Hospital 6	274	5.11	2.69	397	3.55	2.26	139	3.88	2.04

N, total number of data sets; SD, standard deviation

e-Table 2. P values in Mann-Whitney U test.

	Baseline survey versus 2nd quarter	Baseline survey versus last quarter	2nd quarter versus last quarter
Hospital 1	0.000	0.000	0.000
Hospital 2	0.000	0.040	0.024
Hospital 3	0.001	0.011	0.284
Hospital 4	0.000	0.000	0.583
Hospital 5	0.000	0.000	0.000
Hospital 6	0.000	0.000	0.071

One hospital observed the effects of an intervention during the study period, when rofecoxib was replaced entirely by paracetamol (i.e. acetaminophen) after the former was withdrawn from the market due to safety concerns. A comparison of outcomes three months before and three months after the change revealed an increase in maximum pain intensity from 2.2 to 3.8 (NRS), as well as in the proportion of patients with pain on movement (i.e. from 24% to 39%) or with a pain-related sleep disturbance (i.e. from 14% to 32%) (P < 0.05).

The lowest mean score for the outcome measure "Maximum pain intensity since surgery" was reported by patients who had received surgical procedures in gynecology and urology; the highest mean score was reported by patients who had received surgical procedures in orthopedics/traumatology. An overview of the different outcome measures, broken down according to surgical disciplines, is given in table 2. Patients rated orthopedic and traumatological procedures as being among the most painful operations (table 3). Laparoscopic appendectomy, a relatively minor abdominal procedure, was also regarded as exceptionally painful. In comparison, patients rated the intensity of pain after large intestine resection as being considerably lower (numerical rating scale: 4.5 ± 2.9).

Table 2. Outcome measures broken down according to surgical discipline.

Outcome measure	General surgery	Orthopedics/ traumatology	Gynecology/ urology	Other
Maximum pain intensity*1	4.1 ± 2.6	4.5 ± 2.7	3.3 ± 2.8	3.9 ± 2.5
Impaired mobility*2	55%	60%	39%	46%
Impaired breathing*2	48%	9%	33%	33%
Impaired sleep*2	27%	36%	21%	34%
Drowsiness as side effect*2	46%	50%	56%	47%
Nausea as side effect*2	23%	17%	21%	22%
Desire for more pain medication*2	12%	16%	9%	9%
Satisfaction*1	12.5±2.5	12.4±2.4	13.0±2.3	12.5±2.7

^*1 Mean and standard deviation on numeric rating scale; ^*2 Percentage of "yes" replies

Table 3. Ranking of most painful surgical procedures.

Maximum pain intensity since surgery (mean)	Surgical procedures (OPS-Code [German surgical procedure code as part of DRG system])
6.0 ± 2.11	Cruciate ligament reconstruction (5-813.4; n = 102)
5.8 ± 2.44	Forearm fracture osteosynthesis (5-794; n = 158)
5.8 ± 2.63	Knee arthroplasty, partially cemented (5-822.12; n = 114)
5.2 ± 2.33	Laparoscopic appendectomy (5-470.1; n = 271)
4.9 ± 3.26	Laparoscopic sigmoid resection (5-455.75; n = 25)

The values given are the mean for the outcome measure "Maximum pain intensity since surgery" (on an 11-point numeric rating scale) for all surgical procedures that were performed at least 25 times in our database (n = 12 389) and in no fewer than half of the participating hospitals.

Correlation between process indicators and outcome measures: We were unable to observe any correlation between the process indicator "pain documentation" and the outcome measures evaluated in this project. Taking account of the entire data set, the corresponding correlation coefficients were below 0.2 (–0.02 to 0.1) for all of the measures. In contrast, there was a weak to medium correlation between the measure "Desire for pain medication" and almost all outcome measures ("Maximum pain intensity since surgery": correlation coefficient 0.30; "pain-related sleep disturbance": 0.34; "pain-related depressed mood": 0.37; "satisfaction": –0.32; for all analyses: P < 0.01).

Discussion

This project provides an important tool for improving the quality of patient care. It stands out because of its strict focus on outcomes from the patient perspective, its utility in everyday practice, its ability to provide immediate outcome feedback, and its use of external benchmarking. The authors were able to identify improvements in the quality of care in the majority of participating hospitals.

Our assessment of selected quality criteria from the questionnaire indicates that it is an appropriate tool for providing a picture of postoperative pain intensity and can pinpoint differences between disparate groups of patients. As such, the project goes beyond the common practice of equating improved outcomes with pain reduction alone. Taking account of functional impairments, as well as the side effects of pain treatment, is essential to achieving a comprehensive description of quality (12, 13). To date, little attention has been paid to these side effects (e.g. vomiting induced by opiates), and systematic data are almost entirely lacking. This can lead to imprecise assessments of overall quality and/or to improper treatment.

The majority of participating hospitals were able to improve outcomes during the project; nevertheless, our findings also make clear that participation, in and of itself, did not automatically lead to an improvement. Nevertheless, if institutions are willing to make changes, this project can serve as an important tool for identifying problem areas and managing quality improvement measures. Here it should be noted that lasting improvements in outcomes are apparently more difficult to achieve than short-term changes.

In this context, QUIPS is especially well suited as a supplement to quality improvement initiatives that follow a certification approach (which often emphasizes structure and process). This raises the question of how this kind of transformation process can be initiated, optimized, and sustained. Taking part in the benchmark meetings was optional, but the open discussions of project results and patient outcomes were regarded by all participants as an excellent model for mutual learning and exchanging experiences. Planning and conducting the meetings, however, was time-consuming and costly. As a result, we have recently developed a web-based platform for presenting concepts generated both by hospitals with good outcomes (best practice) and by hospitals with poor outcomes (learning from mistakes), the latter of which are presented in an anonymous fashion.

By reflecting changes in quality after a switch in medications, the project showed that it is able to provide additional information on the effects of interventions in everyday clinical practice. Naturally, methodological limitations (e.g. potential covariance, and group comparability) need to be kept in mind. This kind of approach cannot, and should not, replace randomized controlled trials (RCTs), but rather assess whether they can be translated into everyday clinical reality—an issue that normally is considered either not at all or only within the framework of subjective, personal assessments.

As expected, the comparison of postoperative pain intensity according to surgical discipline and surgery type revealed large differences. A surprising finding, however, was that procedures commonly thought of as routine, such as laparoscopic appendectomy, were rated by the patients as being especially painful. This indicates that QUIPS can help identify examples of inadequate care or especially high-risk patient collectives for which hospitals should reconsider or improve established treatment methods.

The presence of specific processes, such as routine pain documentation (recommendation grade A in the AWMF guidelines 041/001 [8]), was not inevitably associated with improved outcomes. In contrast, another process—the availability of pain medication, which can be achieved by means of appropriate algorithms, nurse training, or the use of patient-controlled analgesic systems—appears to have a considerably stronger association with the various outcomes.

These findings show that process indicators are only surrogates and should be continuously evaluated to establish whether they truly lead to improved outcomes. Our findings also highlight the limited extent to which the results of RCTs can be transferred to routine care and future situations, since they represent a snapshot of a highly selected population and/or a specific setting (17).

This project is limited by the fact that participating institutions were not a representative cross-section of German hospitals, and their participation in the project, in itself, may signal that they represent a particularly motivated group. This, however, is a common limitation in studies of quality improvement measures. Despite the stratification of our findings according to surgical discipline, it would have been helpful to take into account the comparability of influencing covariables (e.g. different spectrum of surgical procedures; comorbidities). Finally, it is important to remember that associations between variables do not necessarily imply causality, but rather can provide information about possible cause-and-effect relationships. These associations become more likely as covariance decreases. This is the case, for example, when a drug or procedure is introduced on a hospital-wide basis on a pre-determined date.

An important goal of the project was to ensure the utility of the survey instrument in everyday clinical practice. By restricting the survey to a random sample of patients and to selected outcome measures, it was possible to limit the time needed per data set to an average of 12 minutes.

After the end of the funding period, the project was taken over by the German Society of Anesthesiology and Intensive Medicine/Professional Association of German Anesthetists, together with the German Surgical Society/Professional Association of German Surgeons.

In order to participate in the project and have access to the member area (i.e. download area; presentation of concepts used by high- and low-performance hospitals) each hospital pays a service charge of 1000 per year (www.quips-projekt.de). Currently, more than 60 hospitals are taking part in QUIPS, and the database consists of more than 46 000 data sets. Many of the methodological aspects of the project (e.g. internet-based, configurable feedback functions; benchmark meetings), as well as its implementation strategies, can be applied in other areas of medicine.

Conclusion

QUIPS is the first quality assurance system to collect and analyze data on outcomes and processes in postoperative pain management under standardized conditions and to provide real-time feedback and interhospital benchmarking. The results of the pilot phase show that the project can help to improve outcomes measurably, and that outcomes are not always strongly correlated with traditional process indicators. Many aspects of the project are applicable to other areas of medicine.