What constitutes a clinically important pain reduction in patients after third molar surgery?

Abstract

BACKGROUND:

For patients with surgical third molar removal, it is unknown what constitutes a clinically important change in patients’ visual analogue scale (VAS) reports of pain intensity.

OBJECTIVES:

To determine what constitutes a clinically important change in pain intensity on a VAS following surgical removal of the third molar.

METHODS:

The study population consisted of patients participating in three randomized trials. Patients were asked to rate their pain three times per day over a period of seven days on a 100 mm VAS after surgical removal of the third molar. Global Perceived Effect was measured on day 1 and day 7 and was used as the external criterion for assessing clinically important pain reduction. Global Perceived Effect scores of 6 (‘much improved’) or higher were classified as clinically ‘successful’, and scores of 5 (‘slightly improved’) or below were classified as clinically ‘unsuccessful’. For each trial, the mean absolute and relative changes in VAS scores were calculated for both ‘successful’ and ‘unsuccessful’ treatments. Sensitivity and specificity analyses were performed.

RESULTS:

The patients who reported ‘successful’ pain reduction showed a relative pain reduction of ≥69% and an absolute pain reduction >2.5 cm on the VAS, whereas patients who classified their pain reduction as ‘unsuccessful’ had a relative pain reduction of ≥18.5% and an absolute pain reduction <0.5 cm on the VAS. Furthermore, sensitivity and specificity analyses showed that a cut-off point of ≥50% relative pain reduction exhibited the best balance of sensitivity and specificity.

CONCLUSION:

Relative pain reduction of ≥50% and an absolute pain reduction of ≥2.5 cm on the VAS were most accurate in predicting a successful pain reduction after a given treatment.

In oral surgery, the removal of third molars is the most common surgical procedure performed in the Dutch National Health System. Reasons for third molar surgery include carious lesions on an adjacent tooth or the third molar, pericoronitis, or the presence of a cyst or tumour. The removal of third molars is a prophylactic treatment against future complications. However, the procedure may be associated with considerable postoperative complications such as infection, dry socket, damage to adjacent teeth, nerve damage, swelling, trismus, hemorrhage, oroantral communication and pain (1–6).

In studies investigating the removal of the third molar, pain assessment is commonly used by clinicians as an index of treatment success, on the basis that patients use pain as a cue for treatment outcomes. A variety of pain rating scales are used to assess patients’ pain outcomes (7–10). The visual analogue scale (VAS) is most commonly used to assess pain intensity. A treatment is defined as successful when patients report postoperative pain that is significantly less than baseline measures or control group measures (7–10). Alternatively, several authors define a pain reduction of 30% to 50% as successful (11–13). In patients with chronic pain, a difference in pain intensity of >33% proved to be diagnostic of clinically important improvement (14). To our knowledge, less information is available concerning this subject in acute pain patients. Furthermore, a clinically relevant pain reduction in patients undergoing removal of the third molar remains undefined, and little is known about this topic. This may lead to misinterpretation of the results of studies regarding third molar surgery because it is not clear what degree of pain reduction is clinically relevant.

The purpose of the present study was to determine the degree of change in third molar removal patients’ VAS and Global Perceived Effect (GPE) ratings that constitutes a clinically important pain reduction.

METHODS

Patients

A total of 220 patients who participated in three different trials investigating surgical removal of the third molars at the VU University Medical Center in Amsterdam (Netherlands) provided pain intensity ratings three times per day for seven days after surgery. In trials I and II, the effect of ice compression versus compression alone was investigated. Trial III analyzed the analgesic effect of epinephrine in patients with surgical removal of the third molars. The results of the effect of ice compression on pain after mandibular third molar surgery are described elsewhere (9), and the results of the other two trials are in preparation. Because the treatments used in each trial differed, the study samples were not merged into one group. There were no dropouts in these three trials. However, for various reasons, several patients were excluded from the analysis. Five participants were excluded from trial I: two did not return the questionnaires; one underwent bilateral third molar surgery (unlike the rest of the participants who underwent surgery for the removal of one mandibular third molar); and two patients returned the pain diary two months after the treatment. Two participants were excluded from trial II because they did not return their questionnaires. Three participants were excluded from trial III: one did not return the questionnaire; and an additional two participants did not return their questionnaires until after the data had been collected and analyzed. The total excluded patients represented <5% of the study group and, therefore, likely did not bias the findings. Patient characteristics are summarized in Table 1.

TABLE 1.

Characteristics of patients in the three trials

Characteristic	Trial I (n=95)	Trial II (n=63)	Trial III (n=62)
Age, years	26.78±9.54	30.01±7.92	24.91±5.42
VAS score, day 1 to day 2	30.3±19.7	33.4±23.7	35.0±17.4
VAS score, day 7	17.0±20.1	17.8±18.9	12.5±18.3
Sex, n
Male	47	25	25
Female	48	38	37

Data presented as mean ± SD unless otherwise indicated. VAS Visual analogue scale

Procedure

To investigate pain reduction, all patients were asked to rate their pain three times per day for seven days on a 100 mm VAS anchored by two extremes of pain: ‘no pain’ on the left end and ‘the worst possible pain’ on the right end. Hence, a total of 21 pain reports were collected per patient. In addition, to be able to compare the clinical meaning of pain reported in the first two days after surgery relative to pain reported at the end of day seven, all patients completed the GPE scale on day 2 and day 7 after surgery. The GPE scale asks participants to provide their evaluations of their recovery following treatment using the following seven options: 1 = worst ever; 2 = much worse; 3 = a little worse; 4 = no change; 5 = a little improved; 6 = much improved; and 7 = best ever. The GPE has previously been validated and was translated from English into Dutch (15).

Statistics

GPE ratings of 6 and 7 were clustered to reflect ‘successful’ treatment or clinically important change in pain intensity. GPE scores of 1 to 5 were defined as ‘unsuccessful’ treatments – changes in pain intensity that are not clinically important or in some cases may be negative (15). As such, a score of 5, reflecting ‘a little improvement’, was classified as an unsatisfactory or ‘unsuccessful’ change in condition following treatment, and it was determined that only scores of 6 (‘much improved’) or 7 (‘ best ever’) constituted a clinically important treatment outcome. After classifying cases as being ‘successful’ or ‘unsuccessful’, the mean change in each groups’ pain ratings on the VAS were calculated based on the difference between participants’ mean pain intensity ratings over the first two days of the recovery period and participants’ mean pain intensity rating on day 7 of the recovery period. Parametric data were analyzed using t tests. A 2 × 2 table was used to calculate the sensitivity, specificity and accuracy values for different cut-off points in absolute and relative pain intensity reduction (14). These pain intensity reductions were compared with the GPE classifications (ie, ‘successful’ and ‘unsuccessful’). Table 2 presents the definition of sensitivity, specificity and accuracy as used in the present study. Sensitivity can be defined as a/(a + c), specificity as d/(b + d) and accuracy as (a + d)/patient population. The best cutoff point was selected by three authors (T Forouzanfar, NE Skorpil and WJJM Martin) and was defined as that which provided the highest overall accuracy coupled with the best balance of sensitivity and specificity. When similar levels of accuracy occurred, the cut-off point with a reasonable value for both sensitivity and specificity was chosen.

TABLE 2.

Calculation of sensitivity, specificity and accuracy

Pain reduction	‘Successful’ patients according to the GPE	‘Unsuccessful’ patients according to the GPE
≥50%	a	b
b <50%	c	d

Sensitivity, a/(a + c); specificity, d/(b + d); accuracy, (a + d)/ patient population. GPE Global Perceived Effect scale

RESULTS

As shown in Table 1, there were no differences between the pain reports of patients in different trials at baseline (P>0.1). Mean absolute and relative VAS difference scores (reflecting mean change in reported pain intensity from days 1 and 2 to day 7 postsurgery) are presented in Table 3 according to trial and treatment outcome (‘successful’ versus ‘unsuccessful’) as judged by participants’ GPE scores. Across all trials, VAS pain intensity ratings were significantly reduced in patients classified as ‘successful’ (P<0.01). A (mean ± SD) relative pain reduction between 69.3±42.0% and 84.8±22.7% and an absolute pain reduction between 2.5±2.2 cm and 2.9±1.6 cm were scored ‘successful’ by the patients.

TABLE 3.

Absolute and relative pain reduction according to the classifications ‘successful’ and ‘unsuccessful’

Trial	Successful		Unsuccessful
	Absolute pain reduction, cm	Relative pain reduction, %	Absolute pain reduction, cm	Relative pain reduction, %
I (n=95)	2.3±1.9	72.3±47.1	−1.1±2.0	−20.5±55.6
II (n=63)	2.5±2.2	69.3±42.0	0.5±2.1	−0.6±75.3
III (n=62)	2.9±1.6	84.8±22.7	0.5±13.3	18.5±45.0

Data presented as mean ± SD. For ‘successful’ patients the pain intensity, according to the visual analogue scale, was reduced significantly in all three trials (P<0.01). ‘Successful’ is defined as ‘much improved’ and ‘best ever’ according to the Global Perceived Effect scale; ‘unsuccessful’ is defined as ‘a little improved,’ ‘not changed,’ ‘a little worse,’ ‘much worse,’ and ‘worst ever’

Figure 1 illustrates the frequency of ‘successful’ and ‘unsuccessful’ treatments reported by patients in trial I whose VAS ratings improved by <0%, 0% to 20%, 20% to 40%, 40% to 60%, 60% to 80% and 80% to 100%. Trials II and III yielded similar tendencies. The difference between the mean VAS score of day 1 and day 2, and the VAS score on day 7 was compared with the GPE (‘successful’ or ‘unsuccessful’) on day 7. ‘Unsuccessful’ and ‘successful’ patients are present in all improvement ranges. However, in the ranges 60% to 80% and 80% to 100%, there were significant more ‘successful’ patients then in the other ranges (P<0.05).

Figure 1).

The frequency distribution of visual analogue scale (VAS) scores associated with ‘successful’ and unsuccessful’ pain reduction according to the Global Perceived Effect scale at day 7

In Table 4, the calculated sensitivity, specificity and accuracy values at various cut-off points are listed. A cut-off point of ≥50% relative pain reduction showed the highest accuracy with the best balance of sensitivity and specificity. In trial I, this cut-off point resulted in a sensitivity of 82.2% and a specificity of 82.1%, with an accuracy of 83.1%. In other words, ‘successful’ patients on the GPE provided an 82.2% likelihood of reporting a relative pain intensity reduction of ≥50%, whereas ‘unsuccessful’ patients had an 82.1% likelihood of reporting <50% pain reduction. The calculation for the absolute pain reduction indicated a reduction of ≥2.5 cm as measured on the VAS was an accurate pain reduction.

TABLE 4.

Sensitivity, specificity and accuracy of various cut-off points for relative and absolute pain reduction, compared with the Global Perceived Effect Scale

	Trial I						Trial II						Trial III
Relative pain reduction	≥20	≥30	≥40	≥50	≥60	≥70	≥20	≥30	≥40	≥50	≥60	≥70	≥20	≥30	≥40	≥50	≥60	≥70
Sensitivity, %	89.5	89.3	85.7	82.2*	80.6	75.0	88.6	82.7	82.7	82.7*	74.3	57.1	95.6	95.6	95.6	95.6*	91.1	80.0
Specificity, %	64.1	66.8	74.4	82.1*	88.9	89.7	50.0	57.1	75.0	85.7*	85.7	89.2	52.9	58.2	82.3	82.4*	82.4	82.4
Accuracy, %	78.9	80.0	81.0	83.1*	82.1	82.1	71.4	71.4	79.4	84.1*	79.4	71.4	83.9	85.5	90.9	92.9*	88.7	80.7
Absolute pain reduction, cm		≥1	≥2	≥3	≥4	≥5		≥1	≥2	≥3	≥4	≥5		≥1	≥2	≥3	≥4	≥5
Sensitivity, %		69.6*	53.6	35.7	23.2	7.0		71.3*	48.6	37.2	20.0	11.4		86.7*	62.2	48.9	20.2	9.0
Specificity, %		82.1*	84.6	94.9	98.9	99.7		71.4*	85.7	89.3	92.6	92.9		68.8*	87.5	99.8	99.8	99.9
Accuracy, %		74.4*	66.3	60.0	54.7	45.3		71.4*	65.1	60.3	50.8	47.6		80.7*	67.7	58.1	40.3	32.3

^*Values that are the most accurate cut-off points

DISCUSSION

The results of the present study indicate that a relative pain reduction of ≥69% was perceived to be a ‘successful’ reduction in patients who underwent third molar removal. In contrast to ‘unsuccessful’, the pain intensity was reduced significantly in patients classified as ‘successful’. Furthermore, a relative pain reduction of 50% as a cut-off point was determined to be the most accurate cut-off point, with the best sensitivity and specificity. For absolute pain reduction, a reduction of 1 cm on the VAS could be defined as the most accurate cut-off point. Although the accuracy of the cut-off points was very high, the sensitivities for trials I and II may be somewhat low. On the other hand, the cut-off points that were selected showed the best balance among accuracy, sensitivity and specificity, which convinced us to select these values.

Our study compared the GPE with the VAS. The GPE was used as the ‘gold standard’ in this comparison, which may be questionable. The VAS provides an estimate of the patient’s pain intensity. Pain assessment using a VAS score can only indicate the magnitude of a decrease or increase in pain intensity. It provides less information concerning the degree of success of a treatment as defined by patients. Therefore, using a VAS to define a treatment as successful or not would be a physician-based rather than a patient-based judgment. The GPE, in contrast, has the advantage of assessing the degree of treatment success as defined by the patient and, therefore, making any statement patient based. It reflects a qualitative judgement of treatment success. In defining cut-off points, several authors have used different outcomes – eg, an additional dose of rescue medication (14), expert opinion (16,17) or the seven-point Likert scale (18,19) – as the gold standard by which to evaluate the effectiveness of the study intervention. These standards have potential limitations. In using the additional dose of rescue medication as a standard, for example, one must consider the factors other than pain that may play a role in the patient’s decision to take rescue doses. Regarding experts’ opinions, the variability and inaccuracies of observers in assessing patient pain or psychosocial state has been described by several studies (20–24). Comparison of the expert opinion with patients’ reported pain may bias the results or reflect differences of opinion. In the use of the Likert scale, the comparison of that scale with a quality-of-life measurement or pain intensity assessment is a comparison of two subjective scales, and the results may be biased by the patient’s sense of what the interviewer wants to hear or what the patient believes the answer ought to be. The patient (using the Likert scale) may report satisfaction to please the doctor, because they were seen more quickly or simply because the patient’s mood has improved. In contrast, in using the VAS in a clinical setting, the patients may report less pain after treatment, because reporting otherwise would be too cognitively dissonant to be acceptable.

In the present study, two subjective measurements were compared to identify a meaningful pain reduction in patients who underwent third molar removal. The aforementioned limitations must, therefore, be considered in the present study. For our study, the GPE was translated into Dutch, which may have affected our results. Furthermore, the GPE is normally used to compare pretreatment pain with post-treatment pain. In patients with an impacted third molar, there is often no pain before treatment and the postoperative pain decreases gradually. The patients in the present study were asked to compare the pain they experienced during the first two days after surgery with the pain they experienced on day 7. It is possible that patients misinterpreted these instructions, comparing their pain on day 7 with baseline levels of pain, before surgery, thereby creating measurement error. Nevertheless, the results show clearly that, in all trials, pain decreased significantly in ‘successful’ patients. The accuracy of the cut-off points in the trials proved to be comparable.

Despite the shortcomings of the present study, it can be concluded that a relative pain reduction of 50% and an absolute pain reduction of 2.5 cm on the VAS are most accurate in predicting a successful pain reduction.

SUMMARY

The present study investigated what constituted a clinically relevant acute pain reduction for patients who undergo third molar removal. Patients were asked to rate their pain three times per day over a period of seven days on a VAS after surgical removal of the third molar. To compare the pain in the beginning and in the end of the period being studied, patients also completed the GPE scale. Relative pain reduction of ≥50% and an absolute pain reduction of ≥2.5 cm on the VAS were found to be most predictive of participants’ reports of successful pain management after third molar surgery.