Abstract
OBJECTIVE: To clarify the issues of generalizability arising from the use of instrumental variable (IV) methods to estimate treatment effects in nonexperimental medical outcome studies. DATA SOURCE: We generate Monte Carlo data designed to resemble typical data sets where detailed health status information is unavailable and the treatment assignment process is unobserved. The model used to generate our data makes the realistic assumption that unobservable health status characteristics of patients influence the treatment assignment process and the effectiveness of treatment. STUDY DESIGN: We use Monte Carlo data to illustrate the circumstances where IV estimates generalize to an unobservable patient subpopulation and those where IV estimates generalize to the entire patient population represented by the sample used in the analysis. We also simulate the effect of two policy changes that affect practice patterns. Further, we show that IV estimates are useful for predicting the effect of these changes on treatment effectiveness when the subpopulation to which the IV estimate refers is the same or very similar to the population whose treatment status is affected by the policy change. CONCLUSIONS: Health services researchers cannot take for granted that IV estimates generalize to the same population represented by the sample used for analysis. Instead, researchers must rely on their knowledge of clinical practice and theory regarding the treatment assignment process in interpreting their results and in predicting the effect of changes in practice patterns.
Full text
PDF
Images in this article
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Anderson C. Measuring what works in health care. Science. 1994 Feb 25;263(5150):1080–1082. doi: 10.1126/science.8108722. [DOI] [PubMed] [Google Scholar]
- Gould K. L. Invasive procedures in acute myocardial infarction. Are they beneficial? JAMA. 1994 Sep 21;272(11):891–893. [PubMed] [Google Scholar]
- Gross J. S. Catheterization and mortality in elderly patients with acute myocardial infarction. JAMA. 1995 May 3;273(17):1332–1333. doi: 10.1001/jama.1995.03520410025015. [DOI] [PubMed] [Google Scholar]
- Kuller L. H., Detre K. Catheterization and mortality in elderly patients with acute myocardial infarction. JAMA. 1995 May 3;273(17):1331–1333. [PubMed] [Google Scholar]
- McClellan M., McNeil B. J., Newhouse J. P. Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA. 1994 Sep 21;272(11):859–866. [PubMed] [Google Scholar]
- Wennberg J. E. Population illness rates do not explain population hospitalization rates. A comment on Mark Blumberg's thesis that morbidity adjusters are needed to interpret small area variations. Med Care. 1987 Apr;25(4):354–359. [PubMed] [Google Scholar]
- Wennberg J., Gittelsohn A. Variations in medical care among small areas. Sci Am. 1982 Apr;246(4):120–134. doi: 10.1038/scientificamerican0482-120. [DOI] [PubMed] [Google Scholar]
- von Kummer R., Allen K. L., Holle R., Bozzao L., Bastianello S., Manelfe C., Bluhmki E., Ringleb P., Meier D. H., Hacke W. Acute stroke: usefulness of early CT findings before thrombolytic therapy. Radiology. 1997 Nov;205(2):327–333. doi: 10.1148/radiology.205.2.9356611. [DOI] [PubMed] [Google Scholar]