Abstract
BACKGROUND
To improve pain management, the Veterans Health Administration launched the “Pain as the 5th Vital Sign” initiative in 1999, requiring a pain intensity rating (0 to 10) at all clinical encounters.
OBJECTIVE
To measure the initiative's impact on the quality of pain management.
DESIGN
We retrospectively reviewed medical records at a single medical center to compare providers' pain management before and after implementing the initiative and performed a subgroup analysis of patients reporting substantial pain (≥4) during a postimplementation visit.
PARTICIPANTS
Unique patient visits selected from all 15 primary care providers of a general medicine outpatient clinic.
MEASUREMENTS
We used 7 process indicators of quality pain management, based on appropriately evaluating and treating pain, to assess 300 randomly selected visits before and 300 visits after implementing the pain initiative.
RESULTS
The quality of pain care was unchanged between visits before and after the pain initiative (P >.05 for all comparisons): subjective provider assessment (49.3% before, 48.7% after), pain exam (26.3%, 26.0%), orders to assess pain (11.7%, 8.3%), new analgesic (8.7%, 11.0%), change in existing analgesics (6.7%, 4.3%), other pain treatment (11.7%, 13.7%), or follow-up plans (10.0%, 8.7%). Patients (n =79) who reported substantial pain often did not receive recommended care: 22% had no attention to pain documented in the medical record, 27% had no further assessment documented, and 52% received no new therapy for pain at that visit.
CONCLUSIONS
Routinely measuring pain by the 5th vital sign did not increase the quality of pain management. Patients with substantial pain documented by the 5th vital sign often had inadequate pain management.
Keywords: pain, quality of health care, outcome and process assessment (health care), pain measurement, quality indicators
Pain is one of the most common reasons people seek medical care. Acute pain has been reported as a presenting symptom in over 80% of physician visits and chronic pain has been estimated to be under treated in up to 80% of patients in some settings. 1 In a report issued by the Department of Veteran Affairs, chronic nonmalignant pain was cited as afflicting over 34 million people with substantial cost in lost work days. 2 Uncontrolled pain not only results in unnecessary suffering, but compromises the care of underlying diseases and can lead to depression, decreased enjoyment of life, and less productivity. 3 In 1995, the American Pain Society (APS) set out guidelines delineating that a first step in improving the treatment of pain is assessment and recording of patients' reports of pain. 4 The cited impetus for adopting this strategy is captured in the presidential address to the APS in November 1996 by James Campbell, MD, who stated that “… vital signs are taken seriously … if pain were assessed with the same zeal as other vital signs are, it would have a much better chance of being treated properly.” 5
In November 1998, the Veterans Health Administration (VHA) began a national strategy to improve pain management for its patients. 6, 7 In a 1999 memorandum to its over 1,200 sites of medical care, the VHA initiated the measurement and documentation in the electronic medical record of patients' self-report of pain. 8 The initiative called “Pain as the 5th Vital Sign” required use of a Numeric Rating Scale (NRS) for all clinical encounters. 9 The NRS is a standardized 10-integer verbal and/or visual analog scale (0=no pain, 10=worst possible pain) for patient self-report of pain intensity. The initiative required that intake nurses assess pain and intensity for outpatient visits using the NRS and record the result in the medical record with the vital signs. The implementation phase of the pain-as-the-5th-vital-sign (P5VS) initiative included mandatory classroom-based provider education and supplemental materials to support practice change. Beyond the documentation of pain intensity, the initiative expected “… that a pain score of 4 or higher would trigger a comprehensive pain assessment and prompt intervention …” by the healthcare provider. 9
We report an evaluation on the effect of P5VS on pain management at an outpatient general internal medical clinic within the VA health care system. Our project had 2 major aims. First, we compared pain management before and after implementation of the P5VS initiative using explicit quality indicators. Second, we evaluated the quality of pain management provided to patients in the postimplementation group who reported a substantial level of pain.
METHODS
Study Design
To achieve our first aim, we conducted a retrospective cross-sectional medical record review at a single Veteran Affairs medical center. We extracted data from the electronic medical record for random unique patient visits for 1 group before implementation of the initiative (preimplementation group) and for 1 group after implementation of the initiative (postimplementation group).
To achieve the second aim of a quality of care evaluation for veterans with a substantial level of pain, we analyzed the visits in the postimplementation group for patients who reported pain intensity on the NRS greater than or equal to 4. The pain level criterion of 4 was selected based on the guidelines for care established in the P5VS initiative.
We performed this study with a waiver for consent and with approval from the Institutional Review Board.
Sampling Strategy and Data Extraction
The VHA pain initiative was adopted in late 1999 but it was not fully implemented until January 2001 because of modification to the computerized patient record system (CPRS) and provider education. To cleanly separate the groups, we constructed the preimplementation group to consist of randomly selected clinic visits dating from before October 1999 and the postimplementation group to consist of visits dating after January 2001.
We extracted data from CPRS for unique outpatient visits for each of the attending general internal medicine physicians, including only 1 visit per patient. We used a balanced sampling strategy, randomly selecting 20 visits from 100 sequential eligible visits for each and every attending internal medicine physician who practiced in the nontrainee clinic throughout the study period. Data were extracted from provider progress notes, pharmacy fields, order fields, and the vital sign fields of the electronic medical record to include demographic data, pain levels, and clinical information. We measured the quality of care provided for pain management by whether or not documentation of pain assessment, evaluation, or treatment satisfied the criteria for 7 unique explicit process indicators.
Quality Indicators of Pain Management
We developed 7 explicit process indicators on the basis of VHA P5VS initiative toolkit and published standards of care for pain management (Table 1). 3, 9– 12 The indicators were designed to measure providers' attention, evaluation, and treatment of pain. Development of the indicators involved iterative discussions of potential indicators with experts at the Center for Research on Symptom Management in Life-Threatening Illness at the Oregon Health & Science University, School of Nursing. As the first aim of our study was to compare pain management before and after implementation of the P5VS initiative, all indicators were applied to all visits, assuming that equal numbers of random visits would have indications for pain management. We scored indicators as being satisfied if any documentation existed in the medical record to demonstrate the process of care was provided, including notation of the absence of a finding or that a test or therapy was unneeded or unwanted. Three composite scores were created by combining the 7 indicators by evaluation, treatment, or any attention to pain (Table 1).
Table 1.
Explicit Process Indicators for Quality Pain Management
| Explicit Process Indicator | Criterion Definition |
|---|---|
| 1. Subjective assessment | Did MD document subjective assessment in note (y/n) |
| 2. Exam for pain complaint | Did MD document pain examination evaluating pain (y/n) |
| 3. Any orders to assess pain | Did MD order any evaluations to assess pain (y/n) |
| 4. New pain prescription | Did MD order any new medications for pain (y/n) |
| 5. Change in pain prescription | Did MD change any medications for pain (y/n) |
| 6. Any other pain treatment | Did MD order other therapies for pain treatment (y/n) |
| 7. Follow-up plan made | Did MD document a pain follow-up plan (y/n) |
| Composite Scores | |
| Composite evaluation | Any criteria met from indicators 1, 2, or 3 |
| Composite treatment | Any criteria met from indicators 4, 5, or 6 |
| Any attention to pain | Any of the 7 indicators were met |
Statistical Analysis
Interrater reliability was assessed on the first 10% of the extracted records for the 2 independent data extractors and reviews performed by the principal investigator. We compared performance between the pre- and postgroups with χ2 analysis and with logistic models using random effects to account for clustering on providers. As no differences were found between pre- and postgroups, we aggregated data for exploratory models and used fixed effects to control for any group differences. We used multivariate logistic regression to explore for associations to the evaluation or treatment of pain, controlling for clustering within the 15 physicians using random effects. Relative risk computations were done for statistically significant covariates using simulation with 5,000 iterations for 95% confidence interval (CI).
We initially powered the study for the pre-post χ2 comparison to detect a difference of 10% between groups with an α set at 0.05 and a β at 0.20. A post hoc calculation was performed to determine the minimal difference detectable between the groups. 13
As our second aim was to measure the quality of care provided to patients who reported a substantial level of pain (NRS level ≥4 as defined by the VHA initiative), 9 we computed rates for how often criteria were met for the 7 process indicators and 3 summary scores.
RESULTS
Data Extraction
We reviewed a total of 600 records, 300 in the preimplementation group and 300 in the postimplementation groups. Interrater reliability on the first 10% of records revealed agreement across all variables that exceeded 98%, κ levels >0.97. Missing data were less than 0.2%; we performed no imputation for missing values.
Patient Characteristics
We found no differences in patient or visit characteristics between the pre- and postgroups (Table 2). The mean age of patients (aggregated between both groups) was 63 years and the majority of patients were male (94%). Overall, 56% of patients had been prescribed pain medications preceding the study visit (as documented by pharmacy records) and prior pain diagnoses were documented in 64% of the sample. For patients who had pain level recorded in any place in the medical record in either group (nurse or provider notes before P5VS in the pre group, n =94 or the 5th vital sign field in the postgroup, n =247), mean pain level was not different between groups (P =.14).
Table 2.
Patient Characteristics*
| Patient Characteristics | Pregroup (n =300) | Postgroup (n =300) |
|---|---|---|
| Age (y±standard deviation) | 62±14 | 63±14 |
| Caucasian race (%) | 74 | 72 |
| Male gender (%) | 94 | 93 |
| Pain diagnosis before study visit (%) | 61 | 67 |
| Visit with primary care provider (%) | 99 | 100 |
| Pain complaint as reason for visit (%) | 8 | 5 |
| Primary diagnosis (%) | ||
| Cardiovascular disorder† | 30 | 27 |
| Endocrine disorder‡ | 13 | 13 |
| Respiratory disorder§ | 9 | 10 |
| Psychiatric disorder∥ | 3 | 3 |
| Malignancy | 1 | 1 |
| On pain medication before study visit (%) | 59 | 54 |
| Opiate containing medications | 36 | 32 |
| Non-opiate NSAID/acetaminophen | 22 | 21 |
| Pain level recorded in P5VS field (%) | (N/A) | 82 (n =247) |
| Pain level recorded anywhere in record (%)¶ | 31 (n =94) | 82 (n =247) |
| Mean pain level (score on 0 to 10 scale±standard deviation) | 1.8±2.8 (n =94) | 2.4±3.1 (n =247) |
P >.05 for comparisons between pre- and postgroups.
Primarily coronary artery disease, congestive heart failure, hypertension, and dysrythmias.
Primarily diabetes, hyperlipidemia, and thyroid disorders.
Primarily obstructive lung disease, bronchitis, and pneumonia.
Primarily post-traumatic stress disorder and major depressive/psychotic disorders.
Mostly recorded in nurse intake note in pre-group.
NSAID, non-steroidal anti-inflammatory medications; P5VS, pain-as-the-5th-vital-sign.
For those records in the postimplementation group, 82% of patients had their pain level recorded in the 5th vital sign field. Forty-five percent of patients reported any pain and 32% reported considerable pain greater than or equal to 4.
Pre- Versus Postcomparison
The main result of our study is that we were unable to detect any improvements in pain management after P5VS (Table 3). The proportion of medical records meeting the quality criteria for pain management were equivalent across all 7 explicit process quality indictors and composite scores before and after implementation of the P5VS initiative, P >.05 for all comparisons. Post-hoc analysis indicated that we had sufficient power to detect differences greater than 10% between the groups. New or changed opiate prescriptions were documented in less than 1% of patients in each group.
Table 3.
Comparison of Care Between Preintervention and Postintervention Groups
| Process Indicator | Preimplementation Quality Rate (%) (n =300)* | Postimplementation Quality Rate (%) (n =300)* | P-Value for Pre/Post Comparison† |
|---|---|---|---|
| Subjective assessment | 49.3 | 48.7 | .866 |
| Exam for pain complaint | 26.0 | 26.0 | .924 |
| Other orders to assess pain | 11.7 | 8.3 | .171 |
| New pain prescription | 8.7 | 11.0 | .331 |
| Change existing pain meds | 6.7 | 4.3 | .208 |
| Any other form of treatment | 11.7 | 13.7 | .461 |
| Follow-up plan made | 10.0 | 8.7 | .567 |
| Composite evaluation | 50.0 | 50.3 | .867 |
| Composite treatment | 26.0 | 28.3 | .517 |
| Any attention to pain | 52.0 | 53.0 | .802 |
Variables have the potential to be affirmative in 100% of patients, regardless of the presence or absence of pain.
P >.05 for all comparisons between pre and post groups using random effects (to account for physician clustering) logistic regression analysis; post hoc analysis indicated that a minimal difference of 6% to 10% could be detected between the groups.
Quality of Care for Substantial Pain
Seventy-nine patients (32%) in the postimplementation group reported a substantial level of pain greater than or equal to 4 on the NRS. The average pain level in this subgroup was 6.5 (SD 1.9). For these patients with substantial pain, 22% had no attention to pain documented in the medical record by the provider, 27% had no evaluation documented, and 52% received no new or adjusted therapy for pain at that visit (Table 4). We identified only 1 new prescription of opiate-containing medications and 3 dose adjustments to opiate-containing medications among those in this substantial pain subgroup.
Table 4.
Quality of Pain Management for Patients with Substantial Pain (NRS ≥4)
| Process Indicator | Quality Rate (%)* |
|---|---|
| Subjective assessment | 72.2 |
| Exam for pain complaint | 38.0 |
| Any orders to assess pain | 15.2 |
| New pain prescription | 21.5 |
| Change in pain prescription | 11.4 |
| Any other pain treatment | 22.8 |
| Follow-up plan made | 12.7 |
| Composite evaluation | 73.4 |
| Composite treatment | 48.1 |
| Any attention to pain | 78.5 |
Percentage of visits with where quality criteria was met for explicit process indictors and composite scores, n =79.
Associations to Assessment and Treatment of Pain
Across the overall sample, patients who had a provider evaluation were 10 times more likely to have received pain therapy at the visit (P<.0001 by bivariate analysis). There were no differences in evaluation or therapy by individual providers; α levels for bivariate correlation between providers and indicators ranged from P =.27 to P =.57. Models exploring associations to provider evaluation and treatment revealed few relationships (Table 5). Logistic regression suggested increased provider evaluation was associated with pain level and the patient having a prior pain disorder; provider evaluation was negatively related to the patient's age. Provider treatment of pain was associated with pain level, having a visit reason expressly for pain, and the presence of a prior pain disorder.
Table 5.
Regression Results Exploring for Associations to Pain Evaluation and Treatment
| Model 1—Evaluation | |||
|---|---|---|---|
| Covariate | Coefficient in Model* | P-Value | Relative Risk Ratio (95% Confidence Interval) |
| Age of patient at time of visit | −0.005 | .012 | 0.98 (0.97, 0.997)† |
| Male gender | 0.042 | .670 | — |
| Minority race | 0.055 | .649 | — |
| Pain diagnosis before study visit | 0.230 | <.001 | 1.09 (1.05, 1.14) |
| Visit with primary care provider | 0.463 | .239 | — |
| Pain complaint as reason for visit | 0.354 | <.001 | 1.13 (1.06, 1.19) |
| Pain level (any recorded level) | 0.037 | <.001 | 1.06 (1.06, 1.08)‡ |
| Model 2—Treatment | |||
|---|---|---|---|
| Covariate§ | Coefficient in Model∥ | P-Value | Relative Risk Ratio (95% Confidence Interval) |
| Age of patient at time of visit | −0.003 | .085 | — |
| Male gender | −0.071 | .452 | — |
| Minority race | 0.171 | .138 | — |
| Pain diagnosis before study visit | 0.100 | .049 | 1.04 (1.00, 1.09) |
| Visit with primary care provider | 0.300 | .477 | — |
| Pain complaint as reason for visit | 0.260 | .006 | 1.11 (1.03, 1.19) |
| Pain level (any recorded level) | 0.037 | <.001 | 1.06 (1.04, 1.09)‡ |
Logistic regression model using random effects (to account for physician clustering) and fixed effects to account for pre or post group (model R2.21, Wald χ2 94.1,P <.001).
Relative risk ratio for age presented for 10-year increment.
Relative risk ratio for pain level presented for 1 SD pain increment.
Provider evaluation was associated with provider treatment (bivariate r =.50, P <.0001) but was not included in the model as a covariate because it is not a patient-visit level variable.
Logistic regression model using random effects (to account for physician clustering) and fixed effects to account for pre or post group (model R2 .15 , Waldx χ2 60.4, P<.001).
DISCUSSION
We found that the P5VS initiative by itself did not improve the quality of pain management in an outpatient internal medicine clinic as assessed by our 7 basic process indicators. The measurement and documentation of patients' pain level is a necessary aspect of pain management. However, we were unable to detect any improvements in evaluation and treatment of pain in the year after the P5VS initiative was implemented at a single VA institution. Moreover, for patients who had pain levels greater than or equal to 4 at clinic intake, we demonstrated substantial deficits in the evaluation and therapy of pain. Over one-fifth of patients who reported substantial pain had no attention to pain in the medical record by the providers in our study and fewer than half of these patients had therapeutic interventions at the time of their clinic visit. Our investigation of the P5VS initiative at a single VA site has shown the routine documentation of pain levels, even with system-wide support and broad-based provider education, was ineffective in improving the quality-of-care.
The VHA adopted the logical first step of requiring measurement in the P5VS initiative to identify patients with pain, as did the Joint Commission of Accreditation of Healthcare Organizations and others. 11 The underlying assumption is that measurement, identification, and documentation of patients in pain should lead to improved management. 5, 14 However, efficacy trials of this strategy have not been performed to date and our results suggest that other strategies are needed to change provider behavior.
What opportunities for improvement do these data suggest? Of medical records reviewed after the initiation of the P5VS initiative, 82% had a pain level recorded in the vital sign field. Our study documented a lower compliance with recording of the pain 5th vital sign than prior testing with the initiative (Tollett JH. R.E. Research on pain 5th vital sign, July 31, 2002, personal email communication). However, the ineffectiveness of the initiative on quality of care is not likely to be explained simply on the basis of missing 18% of pain measurement opportunities.
Qualitative studies that interview physicians may be able to identify barriers or targets for quality of pain management improvement. Informal discussions with practitioners at the study site and at other VA clinics across the country have suggested to our research team that some practitioners are not aware of the 5th vital sign field or do not have the pain level available at the time of patient interview. A number of practitioners reported that they receive a paper sheet after clinic intake with basic patient information and only the other 4 vital signs for use during the visit. Some providers noted that they did not use the electronic medical record (and hence do not have the possibility of reviewing the pain 5th vital sign) until after the patient had left and they were charting the visit. Such comments suggest that the utility of the pain 5th vital sign may have been undermined by failures in communication that persist even with documentation. These possibilities should be explored formally in future research and quality improvement activities. Our link between physician's subjective documentation and increased treatment suggest that once a provider performs a pain evaluation, there is a greater chance of management.
The P5VS initiative used both an adaptation of the electronic record and provider education to attempt to change provider behavior. Prior studies have shown that such strategies often fail to change provider behavior in the absence of systematic organizational change. 15, 16 Moreover, pain treatment may have unique challenges because of various barriers to effective pain management from both patients and providers. 3, 17 Improvements in the quality of pain management will likely require multimodality quality improvement and provider change strategies. 18 To improve the quality of pain management, the VA and other health care systems should investigate overlaying other methods like audit/feedback and multidisciplinary Plan-Do-Study-Act improvement cycles as well as improved use of information technology such as clinical alerts and decision support. 15, 16, 19 Since 1999, the VHA has broadened the approach to improving pain management consistent with the efforts it has made in improving other areas of care. 7, 20– 25
Although a small effect, our finding that evaluation of pain decreased with increasing age replicates findings of under-identification and under-treatment of pain in older adults from previous studies. 17, 26– 28 Because of the tendency of older adults to under-report pain, a proactive stance by primary practitioners is needed to identify and effectively manage pain. 17
Our finding of deficient care for those with substantial pain in the postimplementation group might be tempered by the finding that 56% of patients were already taking pain medications at baseline. It is conceivable that our examination of a single visit did not adequately reflect the provider's attention and management of pain. However, for those patients who reported substantial pain levels, prompt intervention would be indicated. The initiation of treatment in only 48% of patients at the time of a visit with pain ≥4 is comparable to the recent national study that documented that patients receive only 55% of recommended care on average. 29
There are several limitations to our study. First, there is much more to pain management than the 7 simple criteria evaluated in this study. A more complete study of pain management would evaluate a larger set of process measures and ideally use a measure of palliation to determine if assessment, evaluation, or treatment strategies resulted in changes in outcomes, such as the improvement in patient suffering. 21, 30
Second, our study employed a retrospective medical record review, which carries the risk of missing important performance factors or undocumented clinician behavior. However, several investigations have validated the medical record review method. 31, 32 Further, the P5VS initiative clearly required documentation of pain management as part of the process of care.
Third, the pre-post design opens the possibility that temporal and secular differences may have affected the results independent of the P5VS initiative. However, as no differences were detected, it is unlikely that such influences importantly affected the results or conclusions of the study. Last, our study was performed in a single clinic at 1 institution and may not generalize to other settings. Our results should be verified in other care settings and geographic locations and explore other aspects of provider behavior change include physician characteristics.
CONCLUSIONS
Our evaluation of the P5VS strategy indicates that routine documentation of pain intensity, while necessary for quality care, may not be sufficient by itself to improve the quality of pain management. Pain evaluation and treatment remains an important focus for medical quality improvement. Additional interventions are needed to improve providers awareness of patients' pain and to increase the rates at which they provide appropriate therapy.
Acknowledgments
We are grateful to the following individuals who contributed to the completion and analysis of this project: Molly Delorit, Theresa Demadura, Anne Rosenfeld, Theresa A. Harvath, Lillian Nail, Jonathan Fields, Nancy Perrin, Jane Tollett, Tuyen Hoang, Lisa Rubenstein, Elizabeth McGlynn, and Yelena Kholodenko. This study was supported as a subproject under a grant from the National Institutes of Nursing Research Exploratory Nursing Research Center Grant 1P20NR0780. The work originated at the Portland Veterans Affairs Medical Center and Oregon Health & Science University, Department of Medicine and School of Nursing; the work was performed at the Portland Veterans Affairs Medical Center and the Veterans Affairs Greater Los Angeles Healthcare System.
REFERENCES
- 1.Kirsch B, Berdine H, Zablotsky D, et al. Management strategy: identifying pain as the fifth vital sign. VHSJ. 2000:49–59. [Google Scholar]
- 2.Health Care Advisory Board. Pain Management in the VA and the Private Sector. Washington, DC: The Advisory Board Company; 1998. [Google Scholar]
- 3.Jacox A, Carr DB, Payne R, et al. 1994 Management of Cancer Pain. Clinical Practice Guideline No.9. AHCPR Publication No. 94-0592. Rockville, MD. Agency for Health Care Policy and Research, U.S. Department of Health and Human Services, Public Health Service, March 1994. [Google Scholar]
- 4.American Pain Society Quality of Care Committee. Quality improvement guidelines for the treatment of acute pain and cancer pain. JAMA. 1995;274:1874–80. doi: 10.1001/jama.1995.03530230060032. [DOI] [PubMed] [Google Scholar]
- 5.American Pain Society. Principles of Analgesic Use in the Treatment of Acute Pain and Cancer Pain. 4. Glenview, IL: American Pain Society; 1999. [Google Scholar]
- 6.Veterans Health Administration National Pain Management Strategy. 1998. November 12, 1998. [DOI] [PMC free article] [PubMed]
- 7.Department of Veterans Affairs. 2005 VHA Directive 2003-021: Pain Management. May 2, 2003. Available at http://www1.va.gov/pain_management/docs/VHAPainDirective_03.pdf Accessed January 2005.
- 8.1999 Veterans Health Administration Memorandum: Pain as the Fifth Vital Sign. March 1, 1999. [Google Scholar]
- 9.Veterans Health Administration. 2005 Pain as the 5th vital sign toolkit. October 2000, revised edition. Geriatrics and Extended Care Strategic Healthcare Group, National Pain Management Coordinating Committee. Available at http://www.va.gov/oaa/pocketcard/pain.asp Accessed January 2005.
- 10.American Pain Society Quality of Care Committee. Quality improvement guidelines for the treatment of acute pain and cancer pain. JAMA. 1995;274:1874–80. doi: 10.1001/jama.1995.03530230060032. [DOI] [PubMed] [Google Scholar]
- 11.JCAHO. 2002 Pain Assessment and Management Standards [online]. Available at http://www.jcaho.org/standard/pm Accessed July 16, 2002.
- 12.Doyle D, Hanks GWG, MacDonald N, editors. Oxford Textbook of Palliative Medicine. 2. Oxford, U.K.: Oxford University Press; 1998. [Google Scholar]
- 13.Hintze J. PASS User's Guide. Kaysville, UT: NCSS; 2000. [Google Scholar]
- 14.Yadgood MC, Miller PH, Mathews PA. Relieving the agony of the new pain management standards. Am J Hospice Pall Care. 2000;17:333–41. doi: 10.1177/104990910001700511. [DOI] [PubMed] [Google Scholar]
- 15.Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc J. 1995;153:1423–31. [PMC free article] [PubMed] [Google Scholar]
- 16.Rubenstein LV, Mittman BS, Yano EM, Mulrow CD. From understanding health care provider behavior to improving health care: the QUERI framework for quality improvement. Med Care. 2000;38:I129–41. [PubMed] [Google Scholar]
- 17.Weiner DK. Improving pain management for older adults: an urgent agenda for the agenda for the educator, investigator, and practitioner [comment] Pain. 2002;97:1–4. doi: 10.1016/s0304-3959(01)00421-3. [DOI] [PubMed] [Google Scholar]
- 18.Du Pen SL, Du Pen AR, Polissar N, et al. Implementing guidelines for cancer pain management: results of a randomized controlled clinical trial. J Clin Oncol. 1999;17:361–70. doi: 10.1200/JCO.1999.17.1.361. [DOI] [PubMed] [Google Scholar]
- 19.Demakis JG, McQueen L, Kizer KW, Feussner JR. Quality enhancement research initiative (QUERI): a collaboration between research and clinical practice. Med Care. 2000;38:I17–25. [PubMed] [Google Scholar]
- 20.2005 Opioid therapy for chronic pain—clinical practice guidelines. Available at http://www.oqp.med.va.gov/cpg/cot/ot_base.htm Accessed January 23, 2005. [Google Scholar]
- 21.Department of Veterans Affairs. 2003. VHA Pain Outcomes Toolkit. February 2003. National VA Pain Outcomes Working Group, National VA Pain Management Coordinating Committee.
- 22.Craine M, Kerns RD. Pain management improvement strategies in the Veteran's Health Administration. APS Bull. 2003;13 Available at http://www.ampainsoc.org/pub/bulletin/sep03/article1.htm Accessed February 22, 2005.
- 23.Asch SM, McGlynn EA, Hogan MM, et al. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med. 2004;141:938–45. doi: 10.7326/0003-4819-141-12-200412210-00010. [DOI] [PubMed] [Google Scholar]
- 24.Kerr EA, Gerzoff RB, Krein SL, et al. Diabetes care quality in the Veterans Affairs Health Care System and commercial managed care: the TRIAD study. Ann Intern Med. 2004;17, 141:272–81. doi: 10.7326/0003-4819-141-4-200408170-00007. [DOI] [PubMed] [Google Scholar]
- 25.Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. N Engl J Med. 2003;348:2218–27. doi: 10.1056/NEJMsa021899. [DOI] [PubMed] [Google Scholar]
- 26.Bernabei R, Gambassi G, Lapane K, et al. Management of pain in elderly patients with cancer SAGE study group. Systematic assessment of geriatric drug use via epidemiology. JAMA. 1998;279:1877–82. doi: 10.1001/jama.279.23.1877. [DOI] [PubMed] [Google Scholar]
- 27.Pahor M, Guralnik JM, Wan JY, et al. Lower body osteoarticular pain and dose of analgesic medications in older disabled women: the Women's Health and Aging Study. Am J Public Health. 1999;89:930–4. doi: 10.2105/ajph.89.6.930. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Morrison RS, Siu AL. A comparison of pain and its treatment in advanced dementia and cognitively intact patients with hip fracture. J Pain Symptom Manage. 2000;19:240–8. doi: 10.1016/s0885-3924(00)00113-5. [DOI] [PubMed] [Google Scholar]
- 29.McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–45. doi: 10.1056/NEJMsa022615. [DOI] [PubMed] [Google Scholar]
- 30.Clark ME, Gironda RJ, Young RW. Development and validation of the pain outcomes questionnaire—VA. J Rehab Res. 2003;40:381–96. doi: 10.1682/jrrd.2003.09.0381. [DOI] [PubMed] [Google Scholar]
- 31.Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283:1715–22. doi: 10.1001/jama.283.13.1715. [DOI] [PubMed] [Google Scholar]
- 32.Dresselhaus TR, Luck J, Peabody JW. The ethical problem of false positives: a prospective evaluation of physician reporting in the medical record. J Med Ethics. 2002;28:291–4. doi: 10.1136/jme.28.5.291. [DOI] [PMC free article] [PubMed] [Google Scholar]