Abstract
OBJECTIVE: Development and evaluation of computerized concurrent utilization review (UR) support taking advantage of a clinically rich computerized patient database. DESIGN: The Automated Support System for Utilization Review (ASSURE) applies the Appropriateness Evaluation Protocol (AEP) Day of Care criteria to computerized patient data in the HELP hospital information system. This paper reports the development, verification, and validation of ASSURE. MEASUREMENTS: Implementation correctness was verified by measuring agreement with a nurse reviewer, using separate sample sets for all 20 criteria for a total of 560 current inpatients. Usefulness in detecting inappropriate days of care was validated by two nurse reviewers who were crossed with manual and computer-assisted review methods in a blocked design for 168 current inpatients. Agreement with reviewers, sensitivity, specificity, positive predictive value, and negative predictive value were measured. RESULTS: Agreement was very good for satisfaction of criteria, and good for appropriateness of day of care. A patient day identified by ASSURE as potentially inappropriate would be twice as likely to be judged inappropriate by a reviewer as a randomly selected patient day. Review of the 10% of patient days identified as potentially inappropriate by ASSURE would identify approximately 21% of the inappropriate days of care. CONCLUSION: ASSURE is a clinically useful tool for screening adult acute care patients for inappropriate days of care, and promises to make a major contribution to reducing health care costs. The prognosis for successful routine clinical use is good.
Full Text
The Full Text of this article is available as a PDF (1.4 MB).
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Byrt T., Bishop J., Carlin J. B. Bias, prevalence and kappa. J Clin Epidemiol. 1993 May;46(5):423–429. doi: 10.1016/0895-4356(93)90018-v. [DOI] [PubMed] [Google Scholar]
- Feldstein P. J., Wickizer T. M., Wheeler J. R. Private cost containment. The effects of utilization review programs on health care use and expenditures. N Engl J Med. 1988 May 19;318(20):1310–1314. doi: 10.1056/NEJM198805193182006. [DOI] [PubMed] [Google Scholar]
- Franks P., Clancy C. M., Nutting P. A. Gatekeeping revisited--protecting patients from overtreatment. N Engl J Med. 1992 Aug 6;327(6):424–429. doi: 10.1056/NEJM199208063270613. [DOI] [PubMed] [Google Scholar]
- Friedman E. Hospital uncompensated care: crisis? JAMA. 1989 Dec 1;262(21):2975–2977. [PubMed] [Google Scholar]
- Gertman P. M., Restuccia J. D. The appropriateness evaluation protocol: a technique for assessing unnecessary days of hospital care. Med Care. 1981 Aug;19(8):855–871. [PubMed] [Google Scholar]
- Miller P. L., Sittig D. F. The evaluation of clinical decision support systems: what is necessary versus what is interesting. Med Inform (Lond) 1990 Jul-Sep;15(3):185–190. doi: 10.3109/14639239009025266. [DOI] [PubMed] [Google Scholar]
- Miller P. L. The evaluation of artificial intelligence systems in medicine. Comput Methods Programs Biomed. 1986 Mar;22(1):5–11. [PubMed] [Google Scholar]
- Payne S. M. Identifying and managing inappropriate hospital utilization: a policy synthesis. Health Serv Res. 1987 Dec;22(5):709–769. [PMC free article] [PubMed] [Google Scholar]
- Pryor T. A., Gardner R. M., Clayton P. D., Warner H. R. The HELP system. J Med Syst. 1983 Apr;7(2):87–102. doi: 10.1007/BF00995116. [DOI] [PubMed] [Google Scholar]
- Pryor T. A. The HELP medical record system. MD Comput. 1988 Sep-Oct;5(5):22–33. [PubMed] [Google Scholar]
- Restuccia J. D., Payne S. M., Lenhart G., Constantine H. P., Fulton J. P. Assessing the appropriateness of hospital utilization to improve efficiency and competitive position. Health Care Manage Rev. 1987 Summer;12(3):17–27. doi: 10.1097/00004010-198701230-00006. [DOI] [PubMed] [Google Scholar]
- Rosenstein A. H. Utilization review. Health economics and cost-effective resource management. Qual Assur Util Rev. 1991 Fall;6(3):85–90. doi: 10.1177/0885713x9100600305. [DOI] [PubMed] [Google Scholar]
- Santos-Eggimann B. Hospital utilization reviews under field conditions: potential and improvements. Int J Technol Assess Health Care. 1993 Fall;9(4):514–521. doi: 10.1017/s0266462300005432. [DOI] [PubMed] [Google Scholar]
- Schoolman H. M. Obligations of the expert system builder: meeting the needs of the user. MD Comput. 1991 Sep-Oct;8(5):316–321. [PubMed] [Google Scholar]
- Stead W. W., Haynes R. B., Fuller S., Friedman C. P., Travis L. E., Beck J. R., Fenichel C. H., Chandrasekaran B., Buchanan B. G., Abola E. E. Designing medical informatics research and library--resource projects to increase what is learned. J Am Med Inform Assoc. 1994 Jan-Feb;1(1):28–33. doi: 10.1136/jamia.1994.95236134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Strumwasser I., Paranjpe N. V., Ronis D. L., Share D., Sell L. J. Reliability and validity of utilization review criteria. Appropriateness Evaluation Protocol, Standardized Medreview Instrument, and Intensity-Severity-Discharge criteria. Med Care. 1990 Feb;28(2):95–111. doi: 10.1097/00005650-199002000-00001. [DOI] [PubMed] [Google Scholar]
- Wickizer T. M., Feldstein P. J., Wheeler J. R., McDonald M. C. Reducing hospital use and expenditures through utilization review. Findings from an outcome evaluation. Qual Assur Util Rev. 1990 Aug;5(3):80–85. doi: 10.1177/0885713x9000500303. [DOI] [PubMed] [Google Scholar]
- Wickizer T. M., Wheeler J. R., Feldstein P. J. Does utilization review reduce unnecessary hospital care and contain costs? Med Care. 1989 Jun;27(6):632–647. doi: 10.1097/00005650-198906000-00006. [DOI] [PubMed] [Google Scholar]