Skip to main content
PMC home page
Health Services Research logoLink to Health Services Research
. 2000 Dec;35(5 Pt 1):977–993.

The outcomes of outcomes and effectiveness research: impacts and lessons from the first decade.

D Stryer 1, S Tunis 1, H Hubbard 1, C Clancy 1
PMCID: PMC1089179  PMID: 11130807

Abstract

OBJECTIVE: To assess the outcomes of the Agency for Healthcare Research and Quality's (AHRQ; formerly the Agency for Health Care Policy and Research, AHCPR) first decade of focus on outcomes and effectiveness research (OER) and to identify needs and opportunities for the study of OER in the coming years. DATA SOURCE: Study findings were collected in response to an inquiry by the Center for Outcomes and Effectiveness Research at AHRQ in July 1997 to all principal investigators (PIs) funded between 1989 and 1997. The request was for investigators to identify their "most salient findings" and supply material for up to three slides. STUDY DESIGN: A taxonomy of 11 non-mutually exclusive categories was used to group the investigators' salient findings by characteristics of methodology or purpose. Two health services researchers assigned findings to up to three categories for each discrete study. PRINCIPAL FINDINGS: Responses were received from 61 (64 percent) of the 91 PIs, reporting on 115 studies. Of the 246 category assignments made, descriptive epidemiology was the most common (24 percent), followed by comparative effectiveness (17 percent) and economic assessments (12 percent). Most studies were retrospective analyses of administrative data. Viewed within a conceptual framework for assessing the impact of research, OER has built a solid foundation for future quality improvement efforts by identifying problems, generating hypotheses, and developing new methodologies and has had limited impact on health care policies, practices and outcomes. CONCLUSIONS: OER has had moderate but significant success meeting initial expectations for the field. Challenges for the next generation of OER include advancing from hypothesis generation to definitive studies of effectiveness, and acceleration of the process by which findings effect policy, practice, and outcomes.

Full text

PDF
989

Images in this article

984Figure 1
on p.984

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Albertsen P. C., Hanley J. A., Gleason D. F., Barry M. J. Competing risk analysis of men aged 55 to 74 years at diagnosis managed conservatively for clinically localized prostate cancer. JAMA. 1998 Sep 16;280(11):975–980. doi: 10.1001/jama.280.11.975. [DOI] [PubMed] [Google Scholar]
  2. Altman D. G., Goodman S. N. Transfer of technology from statistical journals to the biomedical literature. Past trends and future predictions. JAMA. 1994 Jul 13;272(2):129–132. [PubMed] [Google Scholar]
  3. Anderson C. Measuring what works in health care. Science. 1994 Feb 25;263(5150):1080–1082. doi: 10.1126/science.8108722. [DOI] [PubMed] [Google Scholar]
  4. Barry M. J., Cockett A. T., Holtgrewe H. L., McConnell J. D., Sihelnik S. A., Winfield H. N. Relationship of symptoms of prostatism to commonly used physiological and anatomical measures of the severity of benign prostatic hyperplasia. J Urol. 1993 Aug;150(2 Pt 1):351–358. doi: 10.1016/s0022-5347(17)35482-4. [DOI] [PubMed] [Google Scholar]
  5. Brook R. H., Kamberg C. J., Mayer-Oakes A., Beers M. H., Raube K., Steiner A. Appropriateness of acute medical care for the elderly: an analysis of the literature. Health Policy. 1990 May;14(3):225–242. doi: 10.1016/0168-8510(90)90037-e. [DOI] [PubMed] [Google Scholar]
  6. Carlisle D. M., Leape L. L., Bickel S., Bell R., Kamberg C., Genovese B., French W. J., Kaushik V. S., Mahrer P. R., Ellestad M. H. Underuse and overuse of diagnostic testing for coronary artery disease in patients presenting with new-onset chest pain. Am J Med. 1999 Apr;106(4):391–398. doi: 10.1016/s0002-9343(99)00051-0. [DOI] [PubMed] [Google Scholar]
  7. Cohen S. J., Halvorson H. W., Gosselink C. A. Changing physician behavior to improve disease prevention. Prev Med. 1994 May;23(3):284–291. doi: 10.1006/pmed.1994.1040. [DOI] [PubMed] [Google Scholar]
  8. Davis D. A., Thomson M. A., Oxman A. D., Haynes R. B. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995 Sep 6;274(9):700–705. doi: 10.1001/jama.274.9.700. [DOI] [PubMed] [Google Scholar]
  9. Epstein A. M. The outcomes movement--will it get us where we want to go? N Engl J Med. 1990 Jul 26;323(4):266–270. doi: 10.1056/NEJM199007263230410. [DOI] [PubMed] [Google Scholar]
  10. Gardner M. O., Goldenberg R. L. The clinical use of antenatal corticosteroids. Clin Obstet Gynecol. 1995 Dec;38(4):746–754. doi: 10.1097/00003081-199538040-00009. [DOI] [PubMed] [Google Scholar]
  11. Goldenberg R. L., Rouse D. J. Prevention of premature birth. N Engl J Med. 1998 Jul 30;339(5):313–320. doi: 10.1056/NEJM199807303390506. [DOI] [PubMed] [Google Scholar]
  12. Goodman S. N., Berlin J., Fletcher S. W., Fletcher R. H. Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Ann Intern Med. 1994 Jul 1;121(1):11–21. doi: 10.7326/0003-4819-121-1-199407010-00003. [DOI] [PubMed] [Google Scholar]
  13. Graff L. G., Dallara J., Ross M. A., Joseph A. J., Itzcovitz J., Andelman R. P., Emerman C., Turbiner S., Espinosa J. A., Severance H. Impact on the care of the emergency department chest pain patient from the chest pain evaluation registry (CHEPER) study. Am J Cardiol. 1997 Sep 1;80(5):563–568. doi: 10.1016/s0002-9149(97)00422-0. [DOI] [PubMed] [Google Scholar]
  14. Gray B. H. The legislative battle over health services research. Health Aff (Millwood) 1992 Winter;11(4):38–66. doi: 10.1377/hlthaff.11.4.38. [DOI] [PubMed] [Google Scholar]
  15. Greenfield S., Sullivan L., Dukes K. A., Silliman R., D'Agostino R., Kaplan S. H. Development and testing of a new measure of case mix for use in office practice. Med Care. 1995 Apr;33(4 Suppl):AS47–AS55. [PubMed] [Google Scholar]
  16. Hannan E. L., Racz M., Ryan T. J., McCallister B. D., Johnson L. W., Arani D. T., Guerci A. D., Sosa J., Topol E. J. Coronary angioplasty volume-outcome relationships for hospitals and cardiologists. JAMA. 1997 Mar 19;277(11):892–898. [PubMed] [Google Scholar]
  17. Leviton L. C., Goldenberg R. L., Baker C. S., Schwartz R. M., Freda M. C., Fish L. J., Cliver S. P., Rouse D. J., Chazotte C., Merkatz I. R. Methods to encourage the use of antenatal corticosteroid therapy for fetal maturation: a randomized controlled trial. JAMA. 1999 Jan 6;281(1):46–52. doi: 10.1001/jama.281.1.46. [DOI] [PubMed] [Google Scholar]
  18. Maklan C. W., Greene R., Cummings M. A. Methodological challenges and innovations in patient outcomes research. Med Care. 1994 Jul;32(7 Suppl):JS13–JS21. [PubMed] [Google Scholar]
  19. Matchar D. B., McCrory D. C., Barnett H. J., Feussner J. R. Medical treatment for stroke prevention. Ann Intern Med. 1994 Jul 1;121(1):41–53. doi: 10.7326/0003-4819-121-1-199407010-00008. [DOI] [PubMed] [Google Scholar]
  20. Matchar D. B., Samsa G. P., Cohen S. J. Should we just let the anticoagulation service do it? The conundrum of anticoagulation for atrial fibrillation. J Gen Intern Med. 1996 Dec;11(12):768–770. doi: 10.1007/BF02598998. [DOI] [PubMed] [Google Scholar]
  21. Mendelson D. N., Goodman C. S., Ahn R., Rubin R. J. Outcomes and effectiveness research in the private sector. Health Aff (Millwood) 1998 Sep-Oct;17(5):75–90. doi: 10.1377/hlthaff.17.5.75. [DOI] [PubMed] [Google Scholar]
  22. Relman A. S. Assessment and accountability: the third revolution in medical care. N Engl J Med. 1988 Nov 3;319(18):1220–1222. doi: 10.1056/NEJM198811033191810. [DOI] [PubMed] [Google Scholar]
  23. Roberts R. R., Zalenski R. J., Mensah E. K., Rydman R. J., Ciavarella G., Gussow L., Das K., Kampe L. M., Dickover B., McDermott M. F. Costs of an emergency department-based accelerated diagnostic protocol vs hospitalization in patients with chest pain: a randomized controlled trial. JAMA. 1997 Nov 26;278(20):1670–1676. [PubMed] [Google Scholar]
  24. Roper W. L., Winkenwerder W., Hackbarth G. M., Krakauer H. Effectiveness in health care. An initiative to evaluate and improve medical practice. N Engl J Med. 1988 Nov 3;319(18):1197–1202. doi: 10.1056/NEJM198811033191805. [DOI] [PubMed] [Google Scholar]
  25. Rydman R. J., Zalenski R. J., Roberts R. R., Albrecht G. A., Misiewicz V. M., Kampe L. M., McCarren M. Patient satisfaction with an emergency department chest pain observation unit. Ann Emerg Med. 1997 Jan;29(1):109–115. doi: 10.1016/s0196-0644(97)70316-0. [DOI] [PubMed] [Google Scholar]
  26. Schein O. D., Steinberg E. P., Cassard S. D., Tielsch J. M., Javitt J. C., Sommer A. Predictors of outcome in patients who underwent cataract surgery. Ophthalmology. 1995 May;102(5):817–823. doi: 10.1016/s0161-6420(95)30952-9. [DOI] [PubMed] [Google Scholar]
  27. Schroeder S. A. Reflections on the challenges of philanthropy. Health Aff (Millwood) 1998 Jul-Aug;17(4):209–216. doi: 10.1377/hlthaff.17.4.209. [DOI] [PubMed] [Google Scholar]
  28. Shea J. A., Healey M. J., Berlin J. A., Clarke J. R., Malet P. F., Staroscik R. N., Schwartz J. S., Williams S. V. Mortality and complications associated with laparoscopic cholecystectomy. A meta-analysis. Ann Surg. 1996 Nov;224(5):609–620. doi: 10.1097/00000658-199611000-00005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Soumerai S. B., McLaughlin T. J., Spiegelman D., Hertzmark E., Thibault G., Goldman L. Adverse outcomes of underuse of beta-blockers in elderly survivors of acute myocardial infarction. JAMA. 1997 Jan 8;277(2):115–121. [PubMed] [Google Scholar]
  30. Steinberg E. P., Tielsch J. M., Schein O. D., Javitt J. C., Sharkey P., Cassard S. D., Legro M. W., Diener-West M., Bass E. B., Damiano A. M. The VF-14. An index of functional impairment in patients with cataract. Arch Ophthalmol. 1994 May;112(5):630–638. doi: 10.1001/archopht.1994.01090170074026. [DOI] [PubMed] [Google Scholar]
  31. Wennberg J. E., Freeman J. L., Culp W. J. Are hospital services rationed in New Haven or over-utilised in Boston? Lancet. 1987 May 23;1(8543):1185–1189. doi: 10.1016/s0140-6736(87)92152-0. [DOI] [PubMed] [Google Scholar]
  32. Wennberg J. E., Freeman J. L., Shelton R. M., Bubolz T. A. Hospital use and mortality among Medicare beneficiaries in Boston and New Haven. N Engl J Med. 1989 Oct 26;321(17):1168–1173. doi: 10.1056/NEJM198910263211706. [DOI] [PubMed] [Google Scholar]

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust

RESOURCES