The Promise of Electronic Health Records to Inform Triage Decisions in Pneumonia

Predicting the future is hard, but physicians do it every day. Patients want to know their prognosis, and resources need to be apportioned to those who will likely benefit from a test or procedure. We cannot admit every patient to the hospital, and we cannot send all admissions to the intensive care unit. Only those who are likely to deteriorate warrant the highest level of care.

But who needs hospitalization? In the course of their clinical training, physicians learn to identify patients who “look sick.” They consider other factors, like vital signs, radiographic findings, lab values, and comorbid conditions. Together they assemble a gestalt that allows them to make clinical decisions. But physicians often overlook important variables. For example, when evaluating a pneumonia patient, of all the lab values, the most prognostic is the blood urea nitrogen (BUN), which, like the red cell distribution width (RDW),¹ is usually ignored.

In contrast to clinical judgment, risk prediction tools consider all available information to arrive at a prediction. While such tools were originally few, the digitalization of electronic health records has allowed these models to proliferate exponentially. Now, anyone with access to a large dataset can define an outcome, choose candidate predictors and build a model, often using machine learning. Tens of thousands of models have been published, predicting everything from stroke to insulin resistance to missed clinic appointments. Building a model is easy, but external validation—demonstrating that the model predicts accurately in another data set—is rare. Demonstrating that the model will change clinical practice is rarer still. Showing that the model improves patient outcomes—priceless. Only a handful of models have been successfully tested in randomized trials.

Efforts to use prediction models to triage patients with community-acquired pneumonia began in the 1990s with the pivotal Pneumonia PORT trial. Using data abstracted by hand from 14,199 patients hospitalized in 1989, investigators devised the Pneumonia Severity Index (PSI), which incorporated 20 variables, including patient age and sex, comorbidities, vital signs, radiographic findings, and laboratory values.² The PSI categorized patients into five levels of severity. Patients with the two lowest levels were unlikely to die within 30 days (< 1%), while those at the highest level of severity had a mortality in excess of 10%. These results were validated in two external cohorts. More importantly, they have been demonstrated in randomized trials to reduce hospital admissions with no adverse effects. Professional guidelines from the American Thoracic Society (ATS) and the Infectious Diseases Society of America (IDSA) recommend that physicians use the PSI to aid in admission decisions.³ This is one of the few prediction models that has gone through all four stages of development.

Why then, is it not used routinely, and why do we need more prediction models? The PSI is complex, each of the 20 variables has a point value, the points must be summed and then compared to a chart to identify the risk level. In contrast, the CURB-65,⁴ developed by the British Thoracic Society, requires only five variables—confusion, urea, respiratory rate, blood pressure, and age—each of which counts for one point. It is easy to remember and calculate without a computer. However, the CURB-65 has not been shown to reduce admissions, and in head-to-head comparisons, it tends to perform worse than the PSI.³ Moreover, neither model was developed to predict the need for ICU admission, and guidelines do not recommend that it be used in this way.

Much has changed since the Pneumonia PORT. In 2025, neither complexity of a model nor the number of variables should be a barrier to use. In an age where every hospital has an electronic health record, and every physician walks around with a supercomputer in hand, there is no reason to rely on simple scores that can be easily remembered. In addition, the advent of electronic health records allows researchers to inexpensively build models which include tens of thousands of patients and consider a much wider range of risk factors. Removing these constraints promises more accurate models which can be easily deployed in clinical practice. In fact, the ATS/IDSA guidelines cite the need to test models which are incorporated in the electronic health record and generate real-time predictions from routinely recorded data. They also note the need for models to assist in triage to the ICU vs. the ward. This decision is important because patients who are initially admitted to the wards and thereafter escalated to the ICU have higher mortality than those admitted directly to ICU.⁵

In this issue of JGIM, Katz et al. report on the internal validation of just such a model.⁶ Using the electronic health records of 3856 admitted to their internal medicine service over a period of 4 years, they compared its performance to that of the CURB-65. The Katz model uses medical diagnoses, based on the work of Elixhauser,⁷ and combines them with clinical data to predict in-hospital death and escalation to a higher level of care. In the validation set, their model had better discrimination than the CURB-65. This is particularly impressive, because the model, originally trained on more than 55,000 patients, was not designed specifically for pneumonia and lacks some important predictors in the CURB-65, including urea and respiratory rate.⁸ The variables it does contain are all discrete data elements available for most pneumonia patients in the ED, allowing automated calculation. These include labs, such as CRP and serum albumin, which may not be routinely ordered in all hospital systems. Before moving on to external validation, the authors should consider adding urea, RDW, and respiratory rate (all discrete elements which are universally available) to improve prediction and perhaps deleting CRP and serum albumin to increase generalizability.

Almost 20% of the patients had a predicted probability of death of < 1%, suggesting they could be suitable for outpatient treatment. However, 40% of these had a risk of escalation in care that was greater than 6.5%, and 14% had a risk greater than 10%. While it is not known what threshold for escalation is acceptable for sending patients home, it may be lower than that. Perhaps more importantly, we do not know how physicians will use this new tool. During the study period, the hospital relied on the CURB-65. Yet, 35% of the patients in this sample had CURB-65 scores of 0 or 1, which would presumably qualify them for discharge. If physicians did not make decisions based on CURB-65, will they do so with the new tool? Moreover, will they use the tool for deciding on triage to the ICU?

This is the new frontier for prediction tools. Understanding physicians’ reactions to prediction models is still in its infancy. On the one hand, it is important that physicians exercise their clinical judgment. No one wants them to blindly follow a decision rule in all cases. On the other hand, they should follow it in most cases, especially if it performs better than physicians overall. After conducting an external validation of their model, the next step will be to implement the tool, either in a randomized trial, which would be preferable, or in an observational study with historical controls. In so doing, the investigators should pay attention to the principles of implementation science,⁹ because many of the pitfalls of instituting models in the EMR have been identified. It is unlikely that all emergency physicians will use the tool, and those that do may not rely on its recommendations. Understanding the drivers of these behaviors will allow for more uniform care and hopefully lead to better outcomes.