Abstract
Recent guidelines developed by the U.S. Food and Drug Administration for the use of patient-reported outcomes discuss the rating of pain and other symptoms at their current level of severity versus rating these symptoms using a recall period, such as the past 24 hours or past week. To explore whether the overall experience of cancer patients is better represented by ratings of current pain or by pain recalled from the past week, we conducted a secondary analysis of Eastern Cooperative Oncology Group data from 1,147 cancer patients who had reported having persistent pain during the past week. Patients used the Brief Pain Inventory (BPI) to rate their current pain along with their pain at its worst, least, and average during the past week. T-tests were used to compare ratings of current pain and pain recalled from the past week. Linear regressions described the extent to which the various pain ratings contributed to overall pain interference, also derived from the BPI. Overall, patients rated their current pain as less severe than their worst or average pain recalled from the past week. Worst pain recalled from the past week contributed most to ratings of pain interference. These findings indicate that ratings of recalled worst pain, rather than ratings of current pain, might better reflect the overall experience of pain and its impact on function in cancer patients with persistent pain. Our results provide information that might guide the choice of recall period for cancer clinical trials with pain as a self-reported outcome.
Keywords: Persistent pain, cancer, patient report, recall period, Brief Pain Inventory, PRO
Introduction
Pain severity has long been considered a core outcome domain in chronic-pain clinical trials.1 Patients are often asked to rate their pain on a scale from 0, meaning no pain, to 10, meaning the worst possible pain. For both patients and clinicians, this usually implies a rating of current pain, at the time the request is made. However, studies have shown that a rating of current pain may underestimate the patient’s overall pain experience. For example, patients may be reluctant to report their current pain during a clinic visit,2,3 or they may underreport current pain to keep the clinician focused on other aspects of their condition during the limited time of a clinic visit.4 Therefore, a single current-pain rating may underrepresent a patients’ actual pain burden and thus may be inadequate for making clinical decisions and judging the effectiveness of treatments in clinical research.
To provide a more representative picture of the patient’s pain status, questionnaires about pain severity and other dimensions of pain often use a recall period (e.g., your pain in the past 24 hours or during the past week) and may also include a severity descriptor (e.g., your pain at its worst, at its least, or on average) within a given recall period. Although the U.S. Food and Drug Administration (FDA) suggests in its recent draft Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims that “It is usually better to construct items that ask patients to describe their current state,”5 the draft guidance also suggests that the choice of a suitable recall period should depend on the specific purpose of the trial, the characteristics of the disease, and the treatment to be tested.
The type of study may determine the length of recall period to use. For example, some epidemiology studies have used long recall periods, such as 10 years,6 six or more months,7,8 or four weeks,9 whereas recall periods for clinical assessment and measuring pain as a clinical outcome typically are shorter, such as the past 24 hours or the past week. The severity descriptor and recall period used are presumed to influence the resulting pain rating.10,11
Pain ratings made with various recall periods and severity descriptors have been compared with pain ratings recorded in daily or hourly diaries. Among pain ratings from different recall periods (current, past 24 hours, or past week), current-pain ratings have shown the least correlation with the estimates of average pain severity taken from either daily or hourly diaries.12–14
In clinical trials where pain reduction is the target of the investigation, patient-reported pain severity is commonly incorporated as a patient-reported outcome (PRO) measure. The 11-point (0–10) rating scale has been strongly recommended as a core outcome measure of pain intensity in both clinical trials and practice applications.1 Investigators selecting pain assessment instruments for clinical trials in which pain is an outcome of interest need to evaluate the rationale for and the appropriateness of the recall period and the descriptor (such as worst, least, average) to be used for the pain rating.
However, no studies providing direct evidence for choosing an appropriate recall period and pain severity descriptor for clinical trials with pain as an outcome have been conducted in response to the FDA’s draft Guidance for Industry Patient-Reported Outcome Measures. Thus, to examine issues around the use of current pain ratings and ratings with a past-week recall, as well as the influence of pain-severity descriptors (worst, average, and least), we conducted a secondary analysis of data from more than 1,000 patients with cancer who were selected because they had reported persistent pain in the past week. We compared the effectiveness of current pain ratings and past-week worst, average, and least pain ratings in representing the actual burden of pain by comparing the various pain severities and the correlations between pain interference and ratings of pain severity.
Methods
Study Subjects
Our study was a secondary analysis of a combined database derived from two Eastern Cooperative Oncology Group (ECOG) studies of the prevalence and treatment of cancer pain.15,16 A total of 1,786 consecutive outpatients with previously diagnosed recurrent or metastatic cancer had participated in the two studies. Of these, 1,147 patients had responded “yes” to a screening question in the original investigation: “Throughout our lives, most of us have had pain from time to time (such as minor headaches, sprains, and toothaches). Have you had pain other than these everyday kinds of pain during the last 7 days?” These patients were deemed to have had “unusual” pain in the previous seven days and were thus eligible to be included in our analysis. This large sample enabled us to look for potential differences in current and recalled pain ratings within subgroups according to age, gender, ethnicity, disease status, and performance status.
Study Procedures
To assess the severity and the impact of pain, the patients participating in the two ECOG studies completed the Brief Pain Inventory10 (BPI) during a regular clinic appointment. The BPI asks patients to rate their current pain as well as their pain at its worst, least, and average during the past week. (Other versions of the BPI, not used in this study, use a 24-hour recall period.) Each item is rated on a 0–10 scale, where 0 = no pain and 10 = pain as bad as you can imagine. Patients also rated how much their pain interfered with their enjoyment of life, level of activity, ability to walk, mood, sleep, work, and relations with others during the past week. For these BPI items, 0 = does not interfere and 10 = interferes completely. Finally, patients were asked to estimate the percentage of pain relief that they were receiving from their pain treatment.
Statistical Analysis
Alpha was set at 0.05 for type I error control in all comparisons. T-tests were used to compare current-pain ratings with pain levels from recalled worst, least, and average pain. To measure the magnitude of the difference, effect sizes were calculated by dividing mean differences between two samples by the pooled standard deviation of the two groups (Hedge’s g).17 Pearson correlations were calculated to examine relationships between pain intensities and interference, and Steiger’s Z tests18 were used to test the significance of differences between those correlations.
Multiple regression analyses were used to determine the extent to which the pain-intensity rating contributed to pain interference once the other ratings were controlled. In this analysis, the mean of the six BPI pain interference items was treated as a dependent variable and each pain rating was added as a predictor of interference in the second step of a regression after the other three pain ratings were entered in the first step. The change in the ratio of the sum of squares between groups to the total sum of squares (R2) and associated significance levels was used to demonstrate the contribution of each pain rating to pain interference. All statistical analyses were performed using SAS (SAS Institute Inc, North Carolina, USA).
Results
Table 1 shows demographic and disease characteristics of the study subjects. Table 2 compares pain levels recalled from the past week with current pain (BPI “pain now”). Worst pain with a past-week recall had the highest rating, and least pain with a past-week recall had the lowest. Ratings of current pain were significantly lower than worst pain and average pain with a past-week recall but were significantly higher than least pain with a past-week recall. However, the effect size of the difference between current pain and past-week least pain was only 0.13, whereas the effect size of the difference between current pain and past-week worst pain was 0.95 and between current pain and past-week average pain was 0.45. The much smaller effect size indicates a smaller difference between current pain and past-week least pain than between current pain and past-week worst or average pain. Further, when we stratified patients by demographic and disease characteristics, we found no differences between current pain and past-week least pain in patients older than 60, patients who were not Hispanic, patients with disease in remission, and patients whose ECOG performance status score was 0 or 4 (Table 3).
Table 1.
Demographic and Disease Characteristics
| Frequency | Percent | |
|---|---|---|
| Age (yrs) | ||
| 18–50 | 290 | 25.39 |
| 51–60 | 281 | 24.61 |
| 61–70 | 345 | 30.21 |
| 71+ | 226 | 19.79 |
| Missing | 5 | |
| Sex | ||
| Male | 484 | 42.20 |
| Female | 663 | 57.80 |
| Race | ||
| White | 390 | 57.18 |
| Hispanic | 131 | 19.21 |
| Black | 146 | 21.41 |
| Other | 15 | 2.20 |
| Missing | 465 | |
| Current disease status | ||
| Active | 962 | 84.91 |
| Remission | 171 | 15.09 |
| Missing | 14 | |
| ECOG performance status | ||
| 0 | 207 | 18.25 |
| 1 | 475 | 41.89 |
| 2 | 304 | 26.81 |
| 3 | 134 | 11.82 |
| 4 | 14 | 1.23 |
| Missing | 13 | |
Table 2.
Pain Severity by Recall Period
| n | Mean (95% CI) | Median | Differs from Current Pain (95% CI) | Effect Size | |
|---|---|---|---|---|---|
| Past-week worst | 1097 | 5.57 (5.41 – 5.74) | 5 | 2.66 (2.42 – 2.89) a | 0.95 |
| Past-week average | 1092 | 4.08 (3.94 – 4.22) | 4 | 1.16 (0.94 – 1.38) a | 0.45 |
| Past-week least | 1105 | 2.58 (2.44 – 2.72) | 2 | −0.34 (−0.56 – −0.12) a | −0.13 |
| Current | 1109 | 2.92 (2.75 – 3.09) | 2 | 0 | - |
CI = confidence interval.
P < 0.0001 by Wilcoxon rank sum test.
Table 3.
Pain Severity by Selected Factors
| Past-Week Worst | Past-Week Average | Past-Week Least | Current | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| n | Mean | SD | n | Mean | SD | n | Mean | SD | n | Mean | SD | |
| Age (Yrs) | ||||||||||||
| 18–50 | 277 | 5.52 | 2.76 | 275 | 3.98 | 2.30 | 282 | 2.52 | 2.36 | 282 | 3.03 | 2.75 |
| 51–60 | 269 | 5.78 | 2.64 | 267 | 4.25 | 2.18 | 269 | 2.77 | 2.33 | 269 | 3.11 | 2.86 |
| 61–70 | 332 | 5.64 | 2.80 | 332 | 4.15 | 2.49 | 333 | 2.64a | 2.53 | 335 | 2.88 | 2.91 |
| 71+ | 213 | 5.26 | 2.89 | 212 | 3.87 | 2.44 | 215 | 2.32a | 2.35 | 217 | 2.58 | 2.68 |
| Sex | ||||||||||||
| Male | 465 | 5.56 | 2.74 | 459 | 4.18 | 2.35 | 467 | 2.58 | 2.46 | 467 | 3.04 | 2.84 |
| Female | 631 | 5.58 | 2.80 | 632 | 4.01 | 2.37 | 637 | 2.58 | 2.38 | 641 | 2.83 | 2.81 |
| Race | ||||||||||||
| White | 370 | 5.41 | 2.74 | 373 | 3.98 | 2.38 | 378 | 2.59a | 2.45 | 379 | 2.79 | 2.77 |
| Hispanic | 124 | 6.03 | 3.05 | 122 | 4.54 | 2.49 | 121 | 3.14 | 2.70 | 122 | 3.66 | 3.28 |
| Black | 144 | 5.51 | 2.99 | 139 | 4.25 | 2.70 | 144 | 2.81a | 2.58 | 143 | 2.89 | 3.22 |
| Other | 14 | 8.14 | 2.03 | 14 | 6.71 | 2.46 | 15 | 4.80a | 2.81 | 15 | 3.93 | 3.31 |
| Current disease status | ||||||||||||
| Active | 926 | 5.69 | 2.74 | 921 | 4.17 | 2.34 | 933 | 2.62 | 2.42 | 937 | 3.00 | 2.85 |
| Remission | 158 | 4.92 | 2.90 | 159 | 3.53 | 2.40 | 159 | 2.35a | 2.34 | 159 | 2.47 | 2.65 |
| ECOG performance status | ||||||||||||
| 0 | 194 | 4.54 | 2.71 | 194 | 3.12 | 2.07 | 196 | 1.89a | 2.08 | 196 | 1.87 | 2.37 |
| 1 | 453 | 5.23 | 2.71 | 453 | 3.76 | 2.22 | 455 | 2.36 | 2.25 | 458 | 2.56 | 2.60 |
| 2 | 297 | 6.21 | 2.69 | 293 | 4.78 | 2.48 | 298 | 3.07 | 2.64 | 298 | 3.62 | 2.91 |
| 3 | 127 | 6.82 | 2.52 | 129 | 5.01 | 2.20 | 130 | 3.25 | 2.44 | 131 | 4.06 | 3.20 |
| 4 | 12 | 6.08 | 2.61 | 11 | 4.45 | 2.46 | 12 | 2.67a | 2.39 | 12 | 3.42 | 2.84 |
SD = standard deviation.
No significant differences were found when compared with current pain level.
Table 4 demonstrates Spearman correlation coefficients between the various pain intensity ratings and the pain interference items. Among all five pain ratings, worst pain recalled from the past week was most highly correlated with each pain interference item as well as the mean of those six items, whereas current and past-week least pain showed the least correlation with pain interference. The order of the ratings (i.e., coefficients from high to low) was the same for all interference items: past-week worst pain, past-week average pain, current pain, and finally, past-week least pain. Steiger’s Z tests showed that except for relations with others, the correlations between past-week worst pain and mean interference, general activity, mood, walking ability, normal work, sleep, and enjoyment of life are significantly greater than the correlations between other pain ratings (current, past-week average, and past-week least) and those interference items.
Table 4.
| Interference Item | Past-Week Worst | Past-Week Average | Current Pain | Past-Week Least |
|---|---|---|---|---|
| Mean interference | 0.65 | 0.60 | 0.53 | 0.45 |
| General activity | 0.62 | 0.56 | 0.49 | 0.41 |
| Mood | 0.57 | 0.49 | 0.42 | 0.35 |
| Walking ability | 0.52 | 0.48 | 0.45 | 0.40 |
| Normal work | 0.53 | 0.50 | 0.46 | 0.38 |
| Relations with others | 0.48 | 0.46 | 0.39 | 0.35 |
| Sleep | 0.46 | 0.42 | 0.40 | 0.30 |
| Enjoyment of life | 0.52 | 0.47 | 0.41 | 0.35 |
All P < 0.0001.
Steiger’s Z tests showed that, except for relations with others, the correlations between past-week worst pain and other interference items (mean interference, general activity, mood, walking ability, normal work, sleep, and enjoyment of life) were significantly stronger than the correlations between other pain ratings and those interference items.
We then conducted four multiple regression analyses to evaluate which pain rating would most contribute to the patient’s BPI ratings of how pain interfered with general life domains (Table 5). Past-week worst pain, current pain, and past-week average pain made statistically significant contributions to predicting interference even when the other ratings were controlled, accounting for an additional 7%, 2%, and 1% of the variance in predicting pain interference, respectively. The contributions of each pain rating were consistent with the results from correlation analysis. The past-week least pain rating did not show a significant contribution when the other three ratings were controlled. These results suggest that ratings of pain at its worst in the past week have the highest predictive value for patients’ ratings of pain interference, followed by ratings of current pain, past-week average pain, and past-week least pain.
Table 5.
Multiple Regression Analysis Examining Each Pain Rating’s Contribution to the Prediction of Total Pain Interference
| Step and Variable | Total R2 | R2 Change | F Change |
|---|---|---|---|
| Step1: current pain, past-week least pain, past-week average pain | 0.38 | 0.38 | 214.45 |
| Step 2: past-week worst pain | 0.45 | 0.07 | 140.00a |
| Step1: past-week worst pain, past-week least pain, past-week average pain | 0.43 | 0.43 | 265.38 |
| Step 2: current pain, | 0.45 | 0.02 | 41.23 a |
| Step1: current pain, past-week least pain, past-week worst pain | 0.44 | 0.44 | 284.41 |
| Step 2: past-week average pain | 0.45 | 0.01 | 8.33 a |
| Step1: current pain, past-week average pain, past-week worst pain, | 0.45 | 0.45 | 288.82 |
| Step 2: past-week least pain | 0.45 | 0 | 0.98 |
P < 0.01.
Discussion
The results of this study may aid clinical researchers in selecting a recall period and severity descriptor for ratings of pain severity. This information may also be helpful in deciding how to ask patients about pain severity in clinical encounters. For certain interventions, such as breakthrough pain or acute procedural or postoperative pain, repeated assessment of current pain probably presents the most useful representation of the pain experience. However, our results suggest that, for patients with persistent pain, ratings made using a recall period and a pain descriptor (such as “worst” pain) may be more informative than ratings of current pain. Although our analysis was limited to patients with chronic cancer pain, research into other painful conditions, such as back pain, has found that ratings of current pain were lower than recalled average and worst pain.19 Ratings of current pain have also been shown to be consistently lower than mean ratings of usual levels of pain, even when different pain scales are used.20
Reasons that patients might minimize ratings of current pain in the clinic include fear of addiction to pain medicine, concern about analgesic side effects, desire to be a “good” patient (not a complainer), and acceptance of pain as an inevitable part of their illness.21 In addition, we found in the current study that, of all four ratings, current pain was closest to recalled past-week least pain and that there was no significant difference between levels of current pain and past-week least pain in certain subgroups, including patients who were older than 60, patients who were not Hispanic, patients with disease in remission, and patients whose ECOG performance status score was 0 or 4. These results suggest that when pain is an outcome in a clinical trial, patient report of current pain (pain now) may only minimally reflect the patient’s pain burden and may not adequately represent the pain experience during a given time period, especially in some patient populations.
Recent consensus meetings have recommended that patients’ physical and emotional functioning should be considered when designing chronic pain clinical trials.1 Among the pain ratings we analyzed, past-week worst pain had the strongest relationship with pain interference by correlation analysis. In addition, regression analysis showed that past-week worst pain contributed the most in predicting pain interference when the other three ratings were controlled in the regression model. This analysis suggests that, at least in cancer patients with persistent pain, recalled worst pain may better reflect the patient’s overall pain experience and its impact on function and might be the more appropriate severity rating choice for clinical trials in patients with persistent pain or patients with cancer or other painful conditions. However, from this analysis, we cannot comment on the effect of the length of the recall period (e.g., past week or 24 hours). Future studies comparing pain ratings from different recall periods are needed.
The study had several limitations. First, this was a secondary analysis of a study not originally designed to compare the representativeness of different pain ratings. Thus, except for one rating per patient of current pain at the time of assessment, we had no additional real-time pain assessments as a standard against which to evaluate the accuracy of recalled pain ratings. Second, additional variables that may have helped to explain our findings (e.g., patients’ current mood status and satisfaction with pain treatment) were not included in the original studies and therefore could not be assessed. Third, the study design of clinical trials evaluating the effectiveness of a treatment may be influenced by the ability of an outcome to detect change. Therefore, longitudinal studies that explore which pain rating is most sensitive to change in pain severity over time are warranted. Finally, we were only able to compare current pain ratings with recall periods of one week. Many current clinical trials use a 24-hour recall period, and a head-to-head comparison of the performance of 24-hour and one-week recall is needed.
Conclusion
In summary, our study of this very large sample suggests that, in cancer patients with persistent pain, worst-pain levels recalled from the past week may be more representative of a patient’s pain burden than is current pain level. At least for chronic pain, ratings of recalled worst pain appear to be a reasonable choice for both clinical practice and clinical trials.
Acknowledgments
This secondary analysis was supported by grants from the Hawn Foundation, Dallas, Texas, and the Public Health Service (CA 026582) awarded to the Department of Symptom Research at The University of Texas M. D. Anderson Cancer Center. The original studies were supported by grants to the Eastern Cooperative Oncology Group from the Public Health Service (CA 021115), the National Cancer Institute, the National Institutes of Health, and the Department of Health and Human Services.
The authors would like to thank Jeanie F. Woodruff, ELS, for editorial assistance.
Footnotes
The authors acknowledge no potential conflicts of interest, including honoraria, travel to conferences, consultancies, stock ownership (excluding publicly-owned mutual funds), equity interests, or sources of research support.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Dworkin RH, Turk DC, Farrar JT, et al. Core outcome measures for chronic pain clinical trials: IMMPACT recommendations. Pain. 2005;113(1–2):9–19. doi: 10.1016/j.pain.2004.09.012. [DOI] [PubMed] [Google Scholar]
- 2.Von Roenn JH, Cleeland CS, Gonin R, Hatfield AK, Pandya KJ. Physician attitudes and practice in cancer pain management. A survey from the Eastern Cooperative Oncology Group. Ann Intern Med. 1993;119(2):121–126. doi: 10.7326/0003-4819-119-2-199307150-00005. [DOI] [PubMed] [Google Scholar]
- 3.Ryan P, Vortherms R, Ward S. Cancer pain: knowledge, attitudes of pharmacologic management. J Gerontol Nurs. 1994;20(1):7–16. doi: 10.3928/0098-9134-19940101-02. [DOI] [PubMed] [Google Scholar]
- 4.Reyes-Gibby CC, McCrory LL, Cleeland CS. Variations in patients’ self-report of pain by treatment setting. J Pain Symptom Manage. 2003;25(5):444–448. doi: 10.1016/s0885-3924(03)00077-0. [DOI] [PubMed] [Google Scholar]
- 5.U.S. Department of Health and Human Services Food and Drug Administration. [Accessed September 7, 2006];Guidance for industry patient-reported outcome measures: Use in medical product development to support labeling claims. doi: 10.1186/1477-7525-4-79. Available at: http://www.fda.gov/cber/gdlns/prolbl.pdf. [DOI] [PMC free article] [PubMed]
- 6.Dawson EG, Kanim LE, Sra P, et al. Low back pain recollection versus concurrent accounts: outcomes analysis. Spine. 2002;27(9):984–993. doi: 10.1097/00007632-200205010-00020. [DOI] [PubMed] [Google Scholar]
- 7.Gureje O, Von Korff M, Simon GE, Gater R. Persistent pain and well-being: a World Health Organization study in primary care. JAMA. 1998;280(2):147–151. doi: 10.1001/jama.280.2.147. [DOI] [PubMed] [Google Scholar]
- 8.Moulin DE, Clark AJ, Speechley M, Morley-Forster PK. Chronic pain in Canada--prevalence, treatment, impact and the role of opioid analgesia. Pain Res Manag. 2002;7(4):179–184. doi: 10.1155/2002/323085. [DOI] [PubMed] [Google Scholar]
- 9.Jensen MK, Sjogren P, Ekholm O, Rasmussen NK, Eriksen J. Identifying a long-term/chronic, non-cancer pain population using a one-dimensional verbal pain rating scale: an epidemiological study. Eur J Pain. 2004;8(2):145–152. doi: 10.1016/S1090-3801(03)00088-0. [DOI] [PubMed] [Google Scholar]
- 10.Cleeland CS. Measurement of pain by subjective report. In: Chapman CR, Loeser JD, editors. Issues in pain measurement. Advances in pain research and therapy. Vol. 12. New York: Raven Press; 1989. pp. 391–403. [Google Scholar]
- 11.Cleeland CS, Ryan KM. Pain assessment: global use of the Brief Pain Inventory. Ann Acad Med Singapore. 1994;23(2):129–138. [PubMed] [Google Scholar]
- 12.de Wit R, van Dam F, Hanneman M, et al. Evaluation of the use of a pain diary in chronic cancer pain patients at home. Pain. 1999;79(1):89–99. doi: 10.1016/S0304-3959(98)00158-4. [DOI] [PubMed] [Google Scholar]
- 13.Jensen MP, McFarland CA. Increasing the reliability and validity of pain intensity measurement in chronic pain patients. Pain. 1993;55(2):195–203. doi: 10.1016/0304-3959(93)90148-I. [DOI] [PubMed] [Google Scholar]
- 14.Jensen MP, Turner LR, Turner JA, Romano JM. The use of multiple-item scales for pain intensity measurement in chronic pain patients. Pain. 1996;67(1):35–40. doi: 10.1016/0304-3959(96)03078-3. [DOI] [PubMed] [Google Scholar]
- 15.Cleeland CS, Gonin R, Hatfield AK, et al. Pain and its treatment in outpatients with metastatic cancer. N Engl J Med. 1994;330(9):592–596. doi: 10.1056/NEJM199403033300902. [DOI] [PubMed] [Google Scholar]
- 16.Cleeland CS, Gonin R, Baez L, Loehrer P, Pandya KJ. Pain and treatment of pain in minority patients with cancer. The Eastern Cooperative Oncology Group minority outpatient pain study. Ann Intern Med. 1997;127(9):813–816. doi: 10.7326/0003-4819-127-9-199711010-00006. [DOI] [PubMed] [Google Scholar]
- 17.Rosenthal R, Rosnow RL. Essentials of behavioral research: Methods and data analysis. 2. New York: McGraw Hill; 1991. [Google Scholar]
- 18.Meng XL, Rosenthal R, Rubin DB. Comparing correlated correlation coefficients. Psychol Bull. 2007;111(1):172–175. [Google Scholar]
- 19.Bolton JE. Accuracy of recall of usual pain intensity in back pain patients. Pain. 1999;83(3):533–539. doi: 10.1016/S0304-3959(99)00161-X. [DOI] [PubMed] [Google Scholar]
- 20.Bolton JE, Wilkinson RC. Responsiveness of pain scales: a comparison of three pain intensity measures in chiropractic patients. J Manipulative Physiol Ther. 1998;21(1):1–7. [PubMed] [Google Scholar]
- 21.Ward SE, Goldberg N, Miller-McCauley V, et al. Patient-related barriers to management of cancer pain. Pain. 1993;52(3):319–324. doi: 10.1016/0304-3959(93)90165-L. [DOI] [PubMed] [Google Scholar]