Skip to main content
NCBI home page
Journal of the American Medical Informatics Association: JAMIA logoLink to Journal of the American Medical Informatics Association: JAMIA
. 1996 Sep-Oct;3(5):349–357. doi: 10.1136/jamia.1996.97035026

Lessons from evaluating an automated patient severity index.

R F Gibson 1, P J Haug 1, S D Horn 1
PMCID: PMC116319  PMID: 8880682

Abstract

OBJECTIVE: To report lessons learned from evaluation of an automated interface between a hospital clinical information system and a severity of illness index. DESIGN: A system was developed to convert coded electronic patient findings from the HELP System at LDS Hospital into the attributes used by the Computerized Severity Index (CSI) to calculate a severity of illness score. Performance was assessed by comparing the automated CSI score with the manual CSI score (from paper chart review) and by evaluating changes introduced by augmenting the manual CSI score with verified patient data discovered by the automated CSI method. MEASUREMENTS: The strengths and weaknesses of each method are presented. RESULTS: The automated CSI score matched the manual CSI score in 61% of the cases. Sources of errors were analyzed. When the automated score was in error, two-thirds of the time it was due to the lack of codes in the HELP system representing CSI concepts; one-third of the time it was due to nurses not using established HELP system codes. Surprisingly, significant problems were also discovered in the manual system, making it difficult to define a "gold standard". CONCLUSIONS: Automated computerized severity indices have great potential for future applicability once their performance exceeds that of the time-consuming manual chart review method. Neither automated nor manual methods are adequate at the present time. This area remains a fertile ground for future research.

Full Text

The Full Text of this article is available as a PDF (1,007.6 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Alemi F., Rice J., Hankins R. Predicting in-hospital survival of myocardial infarction. A comparative study of various severity measures. Med Care. 1990 Sep;28(9):762–775. doi: 10.1097/00005650-199009000-00006. [DOI] [PubMed] [Google Scholar]
  2. Averill R. F., McGuire T. E., Manning B. E., Fowler D. A., Horn S. D., Dickson P. S., Coye M. J., Knowlton D. L., Bender J. A. A study of the relationship between severity of illness and hospital cost in New Jersey hospitals. Health Serv Res. 1992 Dec;27(5):587–612. [PMC free article] [PubMed] [Google Scholar]
  3. Buckle J. M., Horn S. D., Oates V. M., Abbey H. Severity of illness and resource use differences among white and black hospitalized elderly. Arch Intern Med. 1992 Aug;152(8):1596–1603. [PubMed] [Google Scholar]
  4. Conklin J. E., Lieberman J. V., Barnes C. A., Louis D. Z. Disease staging: implications for hospital reimbursement and management. Health Care Financ Rev. 1984;Suppl:13–22. [PMC free article] [PubMed] [Google Scholar]
  5. Gonnella J. S., Hornbrook M. C., Louis D. Z. Staging of disease. A case-mix measurement. JAMA. 1984 Feb 3;251(5):637–644. [PubMed] [Google Scholar]
  6. Horn S. D., Buckle J. M. Severity indices: potential uses in quality measurement. Top Health Rec Manage. 1989 Dec;10(2):45–55. [PubMed] [Google Scholar]
  7. Horn S. D., Sharkey P. D., Buckle J. M., Backofen J. E., Averill R. F., Horn R. A. The relationship between severity of illness and hospital length of stay and mortality. Med Care. 1991 Apr;29(4):305–317. doi: 10.1097/00005650-199104000-00001. [DOI] [PubMed] [Google Scholar]
  8. Iezzoni L. I. A primer on MedisGroups. Pa Med. 1989 Jan;92(1):28–33. [PubMed] [Google Scholar]
  9. Iezzoni L. I., Ash A. S., Coffman G. A., Moskowitz M. A. Admission and mid-stay MedisGroups scores as predictors of hospitalization charges. Med Care. 1991 Mar;29(3):210–220. doi: 10.1097/00005650-199103000-00003. [DOI] [PubMed] [Google Scholar]
  10. Iezzoni L. I., Daley J. A description and clinical assessment of the Computerized Severity Index. QRB Qual Rev Bull. 1992 Feb;18(2):44–52. doi: 10.1016/s0097-5990(16)30506-1. [DOI] [PubMed] [Google Scholar]
  11. Iezzoni L. I., Moskowitz M. A. A clinical assessment of MedisGroups. JAMA. 1988 Dec 2;260(21):3159–3163. doi: 10.1001/jama.260.21.3159. [DOI] [PubMed] [Google Scholar]
  12. Knaus W. A., Wagner D. P., Draper E. A., Zimmerman J. E., Bergner M., Bastos P. G., Sirio C. A., Murphy D. J., Lotring T., Damiano A. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991 Dec;100(6):1619–1636. doi: 10.1378/chest.100.6.1619. [DOI] [PubMed] [Google Scholar]
  13. Knaus W. A., Wagner D. P., Zimmerman J. E., Draper E. A. Variations in mortality and length of stay in intensive care units. Ann Intern Med. 1993 May 15;118(10):753–761. doi: 10.7326/0003-4819-118-10-199305150-00001. [DOI] [PubMed] [Google Scholar]
  14. Maclure M., Willett W. C. Misinterpretation and misuse of the kappa statistic. Am J Epidemiol. 1987 Aug;126(2):161–169. doi: 10.1093/aje/126.2.161. [DOI] [PubMed] [Google Scholar]
  15. Pryor T. A., Gardner R. M., Clayton P. D., Warner H. R. The HELP system. J Med Syst. 1983 Apr;7(2):87–102. doi: 10.1007/BF00995116. [DOI] [PubMed] [Google Scholar]
  16. Reynolds H. N., Haupt M. T., Thill-Baharozian M. C., Carlson R. W. Impact of critical care physician staffing on patients with septic shock in a university hospital medical intensive care unit. JAMA. 1988 Dec 16;260(23):3446–3450. [PubMed] [Google Scholar]
  17. Smith J. S., Jr, Martin L. F., Young W. W., Macioce D. P. Do trauma centers improve outcome over non-trauma centers: the evaluation of regional trauma care using discharge abstract data and patient management categories. J Trauma. 1990 Dec;30(12):1533–1538. [PubMed] [Google Scholar]
  18. Soeken K. L., Prescott P. A. Issues in the use of kappa to estimate reliability. Med Care. 1986 Aug;24(8):733–741. doi: 10.1097/00005650-198608000-00008. [DOI] [PubMed] [Google Scholar]
  19. Stoskopf C., Horn S. D. Predicting length of stay for patients with psychoses. Health Serv Res. 1992 Feb;26(6):743–766. [PMC free article] [PubMed] [Google Scholar]
  20. Stoskopf C., Horn S. D. The Computerized Psychiatric Severity Index as a predictor of inpatient length of stay for psychoses. Med Care. 1991 Mar;29(3):179–195. doi: 10.1097/00005650-199103000-00001. [DOI] [PubMed] [Google Scholar]
  21. Thomas J. W., Ashcraft M. L. Measuring severity of illness: six severity systems and their ability to explain cost variations. Inquiry. 1991 Spring;28(1):39–55. [PubMed] [Google Scholar]

Articles from Journal of the American Medical Informatics Association are provided here courtesy of Oxford University Press

RESOURCES