The Reliability of Remembered Pretreatment Visual Analog Scale Scores among Hand-Surgery Patients

Abstract

Background  In clinical audits in which preoperative visual analog scale (VAS) scores were not recorded, it would be useful if such scores could be re-created at the time of review.

Patients and Methods  We recorded VAS score for pain during the past week before surgery for 245 consecutive hand-surgery patients scheduled for planned surgery during a 6-month period. A total of 30 patients who refused to participate or were unable to respond were excluded. The remaining 215 patients were contacted after 21 months and asked to furnish a new VAS score of the pain they remembered to have had during the last week before surgery. Responses were analyzed with a Bland-Altman plot.

Results  One hundred and thirty-one (61%) of the patients responded. The mean remembered preoperative score was higher than the mean real preoperative score in all diagnosis groups. The mean difference was 10 mm (standard deviation: 22 mm; standard error of the mean: 2 mm). The lower and upper limits of 95% agreement for individual scores were -33 and 53 mm, whereas the lower and upper limits of the 95% confidence interval of the mean were 6 and 14 mm.

Conclusion  It may be possible to predict the mean real preoperative VAS score in groups of patients with accuracy using the remembered preoperative score. In individual patients, remembered preoperative VAS scores are far too inaccurate to be of value. However, real preoperative scores should be used whenever possible.

Introduction

Since its adaption for evaluating pain intensity in the 1970s, the visual analog scale (VAS) has been very widely used to measure both acute and chronic pain. 1 Pain is a subjective experience, and measurement requires self-reporting. VAS is a simple and often used pain-grading tool to measure this.

Many studies have found strong evidence for the reliability and validity of this tool. 1 2 3 4 5 Zampelis et al 6 reviewed the results of hip revision arthroplasty and found that a simple VAS performed equally well as more elaborate patient-reported outcome measures (PROMs).

Optimally, patients should evaluate their change in symptoms and function by completing the VAS both before treatment and at review. When a preoperative score has not been obtained, the question arises whether patients are able to accurately recall their pretreatment score in retrospect. Only a few studies have investigated patients’ ability to re-create their preoperative PROM score months or years after surgery. One example is the Quick disability of the arm, shoulder and hand (quickDASH) questionnaire. 7 8 These studies evaluated their results with correlation tests and concluded that remembered preoperative scores were quite accurate. However, we feel that this method is inappropriate. It has been pointed out that such tests measure the relationship between two variables, not the agreement between them, and also that data that seem to be in poor agreement can produce quite high correlations. 2 9 In two recent papers from our institution, we studied the accuracy of remembered preoperative QuickDASH scores using Bland-Altman 10 plots. 11 12 We found that remembered preoperative QuickDASH scores were far too inaccurate to be of use in individuals, but it seemed possible that they might be of use in groups of patients.

To our knowledge, no study has investigated patients’ ability to recall their preoperative VAS score months or years after treatment. We used a VAS consisting of a 100-mm unhatched horizontal line where the left end is marked “no pain” and the right end “worst imaginable pain.” 4 The aim of this study was to assess the accuracy of remembered preoperative VAS scores in a consecutive series of upper limb surgery patients.

Patients and Methods

All 245 patients scheduled for planned surgery in the upper extremity at our hospital during the period February to June 2015 were asked on the day of surgery to give a VAS for pain during the past week before surgery. Dupuytren’s disease was treated with collagenase injections. We excluded patients with more than one surgery during the observation period. Total 24 patients refused to participate. We were unable to find the correct address for two patients, another had moved abroad, one had died, and two were cognitively impaired and were unable to answer. These 30 patients were excluded from this study.

There were thus 215 patients for study of remembered pretreatment VAS ( Table 1 ). All received a letter with information about the study and two VAS scales. One scale was to describe their current status, whereas the second asked them to answer as they remembered their pain during the week prior to surgery. The two scales were clearly marked and printed on different colored paper to make it easy to differentiate between them. Letters that were returned because they were wrongly addressed were re-mailed to the correct address. Nonresponders were sent new questionnaires after 7 weeks. Patients who had not responded to the second mailing after 4 weeks were contacted by telephone and asked to return the VAS.

Table 1. Main patient diagnoses.

			Incomplete
	Mailed	No reply	VAS	Evaluated
Abbreviations: AS, arthroscopy; TFCC, triangular fibrocartilage complex; VAS, visual analog scale. Note: Other minor: Bursitis, cysts, trigger finger, etc. Other major: Four corner arthrodesis, elbow synovectomy, metacarpophalangeal prosthesis, etc.
Subacromial impingement	36	8	4	24
Carpal tunnel syndrome	40	11	1	28
Thumb basal joint arthrosis	17	5	0	12
Hardware/foreign-body removal	20	12	0	8
Dupuytren’s contr. (collagenase)	9	1	0	8
Benign tumor	7	1	0	6
Ulnar nerve compression	12	7	1	4
Tendon/ligament surgery	9	4	0	5
TFCC injury—AS wrist	7	3	0	4
Ganglion	12	10	0	2
Finger amputation/arthrodesis	8	6	0	2
Distal radial fracture correction	7	6	1	0
Other, major	15	4	0	11
Other, minor	16	6	0	10
SUM	215	84	7	124

Total 131 (61%) out of the 215 patients responded and were available for this study. However, seven of these patients returned only a current status VAS and were excluded leaving 124 patients for analysis. The mean age of these patients at the time of surgery was 52 (range: 9–87) years. There were 70 (56%) women. The mean time between the real preoperative and remembered preoperative scores was 21 (18–25) months.

Statistics

Visual inspection of histograms of the scores was used to verify that the difference between remembered and real preoperative scores was normally distributed. The scores were shown in a scatter plot. A Bland-Altman plot of the scores was also constructed to visualize the difference between remembered and real preoperative scores relative to their mean value. 10 This study was approved by the regional committee for medical and health research ethics for central Norway (ref. 2015/456).

Results

All diagnosis groups reported a higher median remembered score than the real preoperative score ( Table 2 ). For most diagnoses, there was a lower median score at review ( Table 2 ). The differences between remembered preoperative scores and real preoperative scores were normally distributed. The scatter plot ( Fig. 1 ) shows that the scores deviate considerably from the line y = x of perfect agreement whereas the Bland-Altman plot ( Fig. 2 ) shows that there is a systematic error between remembered and real preoperative scores. This bias appears to be highest in the middle range of real preoperative VAS scores. The remembered preoperative VAS score was a mean of 10 mm higher (standard deviation: 22 mm; standard error of the mean [SEM] 2 mm) than the real preoperative score. Lower and upper limits of 95% agreement for individual scores were -33 and 53 mm, whereas the lower and upper limits of the 95% confidence interval of the mean were 6 and 14 mm.

Table 2. Median VAS scores (mm) before treatment (real and remembered scores) and at review.

	Preoperative		At
	Real	Remembered	review
Abbreviation: VAS, visual analog scale. Note: Other minor: Bursitis, cysts, trigger finger, etc. Other major: Four corner arthrodesis, elbow synovectomy, metacarpophalangeal prosthesis, etc.
Subacromial impingement	51	69	8
Carpal tunnel syndrome	52	57	6
Thumb basal joint arthrosis	75	89	20
Hardware/foreign-body removal	13	30	7
Dupuytren’s contr. (collagenase)	13	33	10
Benign tumor	0	3	0
Tendon/ligament surgery	54	70	1
Other, major	17	39	20
Other, minor	28	44	9

Fig. 1.

Scatter plot of the remembered against the real preoperative visual analog scale (VAS) score and the line of perfect agreement.

Fig. 2.

Bland-Altman plot of the difference between the remembered and real preoperative VAS scores plotted against the average of the two scores. The solid line represents the mean difference. The dotted lines represent the lower and upper limits of agreement.

Discussion

The response rate was well above 50%, indicating a representative sample with little risk of response bias. 13 Patients were unaware that they would be asked to remember their preoperative score. This reduces the risk of learning bias. The fact that there was approximately 2 years between obtaining the real and remembered preoperative scores reduces the risk of learning bias. This period of time is also probably typical of the time before review in many follow-up studies. Our patients are an unselected, consecutive series of upper limb surgery patients. Patients were treated for conditions where both high and low preoperative VAS scores were to be expected. Nevertheless, the number of patients is limited, and we cannot be sure that they represent all categories of patients for whom the VAS is used.

In audits in which pretreatment VAS scores have not been recorded it still seems worthwhile to ask patients to try to remember how they were before treatment. One must suppose that the difference between this scores and the one they give for their present status is an indication of how they themselves view the change in their condition. Many might consider this the most important measure of patient satisfaction. However, the convention is to compare the score at review with one obtained before treatment.

The lower and upper 95% limits of agreements of scores of -33 and 53 mm demonstrate that remembered preoperative VAS scores are far too inaccurate to be of use in individual patients. The 95% confidence of the mean score is, however, much narrower than the limits for individual scores. If the systematic error of 10 mm is subtracted from the mean remembered preoperative score of a group of patients, the real preoperative score will with 95% confidence be this score plus/minus 4 mm (SEM × 1.96), a total interval of 8 mm.

Whether or not this inaccuracy is sufficiently small to make this calculated score useful is a matter of judgment. It is pertinent to keep in mind that although one often gains the impression that VAS scores are very precise, to the extent that mean mm scores are sometimes given with a decimal point, in effect, using a scale from 0 to 1,000, 5 14 15 there is ample evidence that this is, in fact, not the case. Anyone who has watched patients marking VAS scales will have observed the often haphazard way in which they do so. Bijur et al 2 obtained 432 paired VAS scores 1 minute apart and found that the span of the 95% limits of agreement between them was 16 mm. Similarly, DeLoach et al 15 studied VAS responses given 20 minutes apart and concluded that any single VAS measurement should be considered as accurate plus/minus 20 mm.

Not all statistically significant differences in VAS score are clinically relevant. The minimal clinically important differences (MCID), sometimes termed minimal important difference, 5 minimum clinically significant difference, 16 clinically important change, 17 clinically important difference, 18 or minimal important change, 19 is the lowest VAS score that corresponds with a simultaneous verbal statement from patients that the pain is slightly worse or better. MCID has in various studies been reported to lie between 10 and 14 mm. 5 16 17 20 21 22

Conclusion

In light of the inaccuracies and uncertainties inherent in the VAS scale itself, we conclude that it may be possible to predict the mean real preoperative VAS score in groups of patients with useful accuracy using the remembered preoperative score. In individual patients, the remembered preoperative VAS scores are far too inaccurate to be of value. However, we should like to emphasize that the real preoperative scores should be used whenever possible.

Funding Statement

Funding This study was funded by the Norwegian University of Science and Technology, NTNU, Trondheim, Norway. The funding body had no influence in this study design and collection, analysis, and interpretation of data and in writing the manuscript.