Validation of a Method to Identify Immunocompromised Patients with Severe Sepsis in Administrative Databases

Abstract

Rationale: Immunocompromised patients are at high risk for developing severe sepsis. Currently, there are no validated strategies for identifying this group of patients in large administrative databases.

Objectives: We set out to define and validate a method that could be used to identify immunocompromised patients with severe sepsis in administrative databases.

Methods: Patients were categorized as immunocompromised based on the presence of International Classification of Diseases, 9th revision discharge diagnosis codes and medication data. We validated this strategy by comparing the discriminatory ability of the search algorithm to that of manual chart review.

Measurements and Main Results: We identified 4,438 patients at a single academic center with severe sepsis using a definition applied to administrative data described by Angus and colleagues. There were 1,185 (26.7%) who were categorized as immunocompromised based on our novel administrative data search strategy. Compared with identification by medical record review, the new administrative data search strategy had positive and negative predictive values of 94.4% (95% confidence interval [CI], 88.8–97.7%) and 94.3% (95% CI, 91.0–96.6%). The sensitivity and specificity were 87.4% (95% CI, 80.6–92.5%) and 97.6% (95% CI, 95.0–99.9%).

Conclusions: Patients who are immunosuppressed are a large subgroup of those with severe sepsis. Following its validation as a search strategy using other large databases, and its adaptation for International Classification of Diseases, 10th revision, this novel method may allow researchers to account for a patient’s immune state when examining outcomes.

Severe sepsis, characterized by systemic inflammation and acute organ dysfunction due to infection, affects 1 to 3 million patients in the United States per year and results in 250,000 to 350,000 in-hospital deaths (1). Patients who are immunocompromised from medical conditions or medications that interfere with normal immune function are at high risk for developing and dying from severe sepsis. In small studies, patients with immunosuppressive conditions represent approximately one-third of patients with severe sepsis (2, 3). However, these findings have not been replicated in large administrative databases, in part because there is not a validated strategy for identifying immunocompromised patients. We set out to define and validate a search strategy that uses International Classification of Diseases, 9th revision (ICD-9) discharge codes and medication data to identify immunocompromised patients in a large administrative database.

Methods

The University HealthSystem Consortium (UHC) is an alliance of 117 U.S. academic medical centers and 300 of their affiliated hospitals. For hospitalized patients at member centers, the UHC amasses demographic data, medication data, and up to 99 ICD-9 discharge diagnosis codes per hospital discharge. UHC regularly performs rigorous quality assessments of their database.

We used a search strategy described by Angus and colleagues to identify 360,319 patients, at least 18 years of age, with severe sepsis in the UHC database from January 1, 2010 to December 31, 2012 (4). These patients are referred to as having Angus-positive severe sepsis. This search strategy has been used by other investigators to categorize patients as having severe sepsis in the UHC database (5). To ensure that the first hospital admission for severe sepsis within the previous year was captured for all patients, we excluded patients who had an episode of severe sepsis between January 1, 2009 and December 31, 2009. In addition, only the first episode of severe sepsis per patient between 2010 and 2012 was included. In the present study, we restricted our analysis to the 4,438 patients (1.2%) who were hospitalized at the University of Chicago Medical Center. The University of Chicago Institutional Review Board approved this study.

We determined the ICD-9 codes associated with immunosuppressive conditions (Table 1). Three types of conditions were considered definitely immunosuppressive: HIV/AIDS, hematological malignancies, or other intrinsic immune conditions. Patients with three other types of conditions were considered immunosuppressed only if they received an immunosuppressive medication during the studied hospitalization: solid malignancies, organ transplantations, and rheumatologic/inflammatory conditions. All patients with a possibly immunosuppressive condition were considered immunocompromised if they received chemotherapy or immune-modulating agents. Additionally, patients with rheumatologic/inflammatory conditions were considered immunocompromised if they received systemic steroids (Table 2).

Table 1.

International Classification of Diseases, 9th revision discharge codes for immunosuppressive conditions

Code	Description
HIV/AIDS
042	HIV disease
Hematological malignancy
200–208	Lymphatic and hematopoietic tissue malignancy
Other immune conditions
279	Disorders of immune mechanism
288.0	Neutropenia
288.1	Functional disorders of polymorphonuclear neutrophils
288.2	Genetic anomalies of leukocytes
288.5	Decreased WBC count
288.8	WBC disease NEC
288.9	WBC disease NOS
289.83	Myelofibrosis
289.89	Blood diseases NEC
289.9	Blood diseases NOS
795.7	Immunological findings NEC
795.79	Nonspecific immune findings NEC & NOS
Solid malignancy
140–199	Organ/system malignant tumors
209	Neuroendocrine tumors
235–239	Neoplasms of uncertain behavior
Organ transplant
996.8	Complications of transplanted organ
v42	Organ transplant status
Rheumatologic/inflammatory
135	Sarcoidosis
277.3	Amyloidosis NOS
277.31	Familial Mediterranean fever
277.39	Amyloidosis NEC
340	Multiple sclerosis
341	Other CNS demyelination
357	Acute infective polyneuritis
422	Acute myocarditis
446	Polyarteritis nodosa et al.
495.9	Allergic alveolitis/pneumonitis NOS
516	Other alveolar pneumonopathy
555–558	Enteritis and colitis
695.4	Lupus erythematosus
710	Diffuse connective tissue disease
711	Arthropathy with infection
712	Crystal arthropathies
714	Rheumatoid arthritis & inflammatory polyarthropathy
720	Inflammatory spondylopathies
725	Polymyalgia rheumatica

Definition of abbreviations: CNS = central nervous system; NEC = not elsewhere classified; NOS = not otherwise specified; WBC = white blood cell.

Where three- or four-digit codes are listed, all associated subcodes were included.

Table 2.

Immunosuppressive medications

Chemotherapeutic agents

Alkylating agents: busulfan; dacarbazine; estramustine phos sodium; altretamine; bendamustine hydrochloride; thiotepa; chlorambucil; cyclophosphamide; ifosfamide; ifosfamide/mesna; mechlorethamine; melphalan; uracil mustard; carmustine; lomustine; streptozocin

Antibiotics: amsacrine; daunorubicin; daunorubicin citrate liposome; doxorubicin; doxorubicin hcl liposome; epirubicin; idarubicin; bleomycin sulfate; dactinomycin; mitomycin; plicamycin

Antimetabolites: methotrexate; pemetrexed; cladribine; clofarabine; fludarabine phos; mercaptopurine; pentostatin; thioguanine; capecitabine; cytarabine (conv); cytarabine (lipo); floxuridine; fluorouracil; gemcitabine;

Antimitotics: eribulin mesylate; ixabepilone; cabazitaxel; docetaxel; paclitaxel; vinblastine; vincristine; vinorelbine

Monoclonal antibodies: alemtuzumab; bevacizumab; cetuximab; gemtuzumab; ibritumomab; ipilimumab; ofatumumab; panitumumab; pertuzumab; rituximab; tositumomab and iodine; trastuzumab

Other: mitoxantrone; brentuximab vedotin; arsenic trioxide; bortezomib; carfilzomib; everolimus; mitotane; porfimer; pralatrexate; sipuleucel-t; sorafenib; temozolomide; vorinostat; erlotinib; gefitinib; tretinoin; romidepsin; dasatinib; imatinib; lapatinib; nilotinib; pazopanib; sunitinib; temsirolimus; bexarotene; aldesleukin; denileukin diftitox; levamisole; amifostine; dexrazoxane; mesna; azacitidine; decitabine; nelarabine; irinotecan; topotecan; asparaginase; pegaspargase; etoposide; etoposide phos; teniposide; procarbazine; carboplatin; cisplatin; oxaliplatin

Immune-modulating agents

Belimumab; denosumab; eculizumab; palivizumab; auranofin; aurothioglucose; gold sodium thiomalate; leflunomide; abatacept; adalimumab; anakinra; certolizumab pegol; etanercept; fingolimod; golimumab; infliximab; interferon alfa-2a; interferon alfa-2b; interferon alfa-n3; interferon alfacon-1; interferon beta-1a; interferon beta-1b; interferon gamma-1b; lenalidomide; natalizumab; peginterferon alfa-2a; peginterferon alfa-2b; pimecrolimus; thalidomide; tocilizumab; ustekinumab; pegademase bovine; alefacept; azathioprine; basiliximab; belatacept; cyclosporine; daclizumab; efalizumab; glatiramer acetate; muromonab-cd3; mycophenolate acid; mycophenolate mofetil; sirolimus; tacrolimus; palifermin

Systemic corticosteroids

Betamethasone; budesonide; dexamethasone; methylprednisolone; methylprednisolone sod succinate; prednisolone; prednisone; triamcinolone

When categorizing the types of immunocompromised conditions in Figure 1, the six groups were mutually exclusive. Patients were first classified as having HIV/AIDS or not. Those patients without HIV/AIDS were then classified as having a hematological malignancy or not. Those patients without HIV/AIDS or hematological malignancies were classified as having other intrinsic immune conditions or not. This process was continued for solid malignancies, organ transplantation, and rheumatologic/inflammatory, in that order.

Figure 1.

A comparison of two methods for identifying immunocompromised patients among the same group of Angus-positive septic patients at University of Chicago Medical Center (n = 421). (A) Search strategy for administrative database using International Classification of Diseases, 9th revision hospital discharge codes and medication data. (B) Manual review of the medical records for the same patients. When categorizing the types of immunocompromised conditions in this figure, the six groups were mutually exclusive.

To validate this identification strategy applied to the UHC database, we randomly selected 421 of the 4,438 patients (9.5%) with Angus-positive severe sepsis who were hospitalized at University of Chicago Medical Center. A physician with clinical expertise reviewed the full hospitalization record for each of these 421 patients. Patients were classified as having an immunosuppressive condition (Table 1) or taking an immunosuppressive medication (Table 2) by chart review (gold standard). The sensitivity, specificity, positive predictive value, and negative predictive value of the search algorithm were determined. All analyses were performed with STATA 13.1 (StataCorp, College Station, TX).

Results

Among 4,438 patients with Angus-positive severe sepsis at the University of Chicago between 2010 and 2012, 1,185 (26.7%) were identified as immunocompromised based on our search strategy that used ICD-9 diagnosis codes and medication data. Among a randomly selected group of 421 patients, we identified 135 (32%) as immunosuppressed by manual chart review. The two approaches to identify immunosuppressed patients are compared in Figure 1. The administrative data search strategy had positive and negative predictive values of 94.4% (95% confidence interval [CI], 88.8–97.7%) and 94.3% (95% CI, 91.0–96.6%), respectively, for identifying immunocompromised patients compared with the gold standard, manual medical record review (Table 3). The sensitivity and specificity of the algorithm were 87.4% (95% CI, 80.6–92.5%) and 97.6% (95% CI, 95.0–99.9%), respectively.

Table 3.

Positive predictive value, negative predictive value, sensitivity, and specificity for the six categories of immunosuppressive conditions with chart review as the gold standard applied to 421 patients with Angus-positive severe sepsis at the University of Chicago

	Positive Predictive Value (95% CI)	Negative Predictive Value (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)
HIV/AIDS	100 (47.8–100)	99.8 (98.7–100)	83.3 (35.9–99.6)	100 (99.1–100)
Hematological malignancy	88.9 (73.9–96.9)	97.9 (95.9–99.1)	80.0 (64.4–90.9)	99 (97.3–99.7)
Other intrinsic immune	71.4 (53.7–85.4)	99.2 (97.7–99.8)	89.3 (71.8–97.7)	97.5 (95.4–98.8)
Solid malignancy + immunosuppressive medication	75.0 (34.9–96.8)	98.5 (96.6–99.5)	54.5 (23.4–83.3)	99.4 (97.9–99.9)
Organ transplant + immunosuppressive medication	95.7 (78.1–99.9)	99.1 (97.3–99.8)	88.0 (68.8–97.5)	99.7 (98.3–100.0
Rheumatologic/inflammatory + immunosuppressive medication	91.7 (73.0–99.0)	97.8 (95.6–99.1)	75.9 (56.5–89.7)	99.4 (97.7–99.9)
Any of the above	94.4 (88.8–97.7)	94.3 (91.0–96.6)	87.4 (80.6–92.5)	97.6 (95.0–99)

Definition of abbreviation: CI = confidence interval.

Data presented as %.

Among the patients whose medical records were manually reviewed, 11 (2.6%) had solid malignancies and received chemotherapy within the month before hospitalization but not during the hospitalization. These patients may have been misclassified as “not immunosuppressed” by the administrative data search strategy because the cytotoxic effects of the chemotherapy likely last for weeks after administration. Of the 16 patients who were categorized as immunosuppressed based on administration of systemic steroids, all received these medications for management of their rheumatologic/inflammatory medical conditions, not due to an alternative diagnosis such as chronic obstructive pulmonary disease.

Overall, the percentages of immunosuppressive conditions and medications at University of Chicago were similar to those at other UHC centers (Table 4). The greatest dissimilarity was that 24% of patients at University of Chicago had solid malignancies compared with 17% at other centers. Patients in the University of Chicago sample had similar types of infections and organ failure as patients at other academic medical centers in the UHC database. The major difference was that 51% of patients at University of Chicago were black compared with 19% at other academic centers.

Table 4.

Characteristics of patients and episodes of severe sepsis by cohort

Variable	Not University of Chicago		University of Chicago w/out Sample		University of Chicago Sample		Total
	No.	%	No.	%	No.	%	No.	%
Encounters	355,881		4,017		421		360,319
Mean age, yr	63.6		61.8		62.2
Sex
Male	183,627	51.6	2,042	50.8	205	48.7	185,875	51.6
Female	172,254	48.4	1,975	49.2	216	51.3	174,446	48.4
Race
White	230,121	64.7	1,398	34.8	149	35.4	231,669	64.3
Black	69,175	19.4	2,060	51.3	210	49.9	71,446	19.8
Other	36,585	10.3	559	13.9	62	14.7	37,206	10.3
Infection categories
Genitourinary infection	134,518	37.8	1,255	31.2	129	30.6	135,903	37.7
Respiratory infection	118,230	33.2	1,222	30.4	127	30.2	119,580	33.2
Wound/soft tissue/bone/joint infection	53,163	14.9	510	12.7	47	11.2	53,720	14.9
Abdominal infection	39,274	11.0	477	11.9	64	15.2	39,815	11.0
Bacteremia	18,782	5.3	338	8.4	33	7.8	19,153	5.3
Device-related infection	14,300	4.0	237	5.9	27	6.4	14,564	4.0
CNS infection	5,363	1.5	70	1.7	6	1.4	5,439	1.5
Endocarditis	4,790	1.3	76	1.9	6	1.4	4,872	1.4
Other/unspecified infections	207,152	58.2	2,554	63.6	271	64.4	209,978	58.3
Mean No. of infection categories per case	1.67		1.68		1.69
Organ failure categories
Renal	211,402	59.4	2,332	58.1	249	59.1	213,984	59.4
Cardiovascular	143,460	40.3	2,004	49.9	216	51.3	145,681	40.4
Respiratory	84,752	23.8	1,082	26.9	113	26.8	85,948	23.9
Coagulation	84,300	23.7	1,214	30.2	121	28.7	85,636	23.8
Central nervous system	43,050	12.1	255	6.3	20	4.8	43,325	12.0
Hepatic	14,257	4.0	179	4.5	21	5.0	14,457	4.0
Mean No. of organ failure categories per case	1.63		1.76		1.76
Immunosuppressive conditions
Solid malignancy	60,014	16.9	955	23.8	106	25.2	61,075	17.0
Rheumatological/inflammatory	40,495	11.4	527	13.1	55	13.1	41,077	11.4
Hematologic malignancy	25,091	7.1	352	8.8	36	8.6	25,479	7.1
Transplant	22,952	6.4	333	8.3	41	9.7	23,326	6.5
Other Immune condition	21,796	6.1	391	9.7	56	13.3	22,243	6.2
HIV	6,014	1.7	60	1.5	5	1.2	6,079	1.7
Cases with ≥1 immunosuppressive conditions	139,836	39.3	1,955	48.7	221	52.5
Immunosuppressive medications
No immunosuppressive medications	237,833	66.8	2,460	61.2	254	60.3	240,547	66.8
Systemic steroids	77,140	21.7	928	23.1	94	22.3	78,162	21.7
Chemotherapy/immune-modulating agents	40,908	11.5	629	15.7	73	17.3	41,610	11.5

Definition of abbreviation: CNS = central nervous system.

Discussion

In this study, we found that 26.7% of patients were immunosuppressed using an automated search strategy of Angus-positive severe sepsis cases at a single academic medical center. This strategy accurately identified these patients as immunosuppressed or not immunosuppressed more than 90% of the time compared with a gold standard of manual medical record review. The high discriminatory ability of the algorithm suggests that the majority of patients who were coded as having immunosuppressive conditions were actively being treated for these conditions. Our results also suggest that medication data are required to accurately estimate the percentage of immunocompromised patients in an administrative database.

It was a concern that some patients with solid malignancies would have received chemotherapy in the weeks leading up to, but not during, an admission for severe sepsis, leading to misclassification. However, among the group whose charts were manually reviewed, only 2.4% might be inappropriately classified as not immunosuppressed because they received chemotherapy immediately before hospitalization and not during hospitalization. Also, a concern was that some patients with rheumatologic/inflammatory conditions would have received steroids for an alternate diagnosis such as chronic obstructive pulmonary disease and thus be misclassified as immunosuppressed. This was not observed in the University of Chicago cohort.

Over the last 10 years, the incidence of severe sepsis has increased, and hospital mortality from this condition decreased despite the lack of new effective therapies (1). During the same time, treatment for many malignancies and inflammatory conditions has been revolutionized by the introduction of new immune-modulating agents. Our method for identifying immunosuppressed patients would allow researchers to determine whether the increasing incidence of severe sepsis parallels an increase in the number of immunosuppressed patients with severe sepsis. In addition, the expected mortality from severe sepsis for a particular cohort is influenced by the percentage of immunocompromised patients. Our algorithm could allow researchers using UHC or similar administrative databases to adjust for the proportion of immunosuppressed patients when comparing mortality rates among hospitals or at a single hospital over time.

Although immunosuppressed patients compose a large proportion of patients with severe sepsis, they are underrepresented in studies examining new ways to treat severe sepsis. These patients are often excluded from studies on the key immune pathways that are associated with poor outcomes from this often fatal syndrome (6). Our finding that approximately one-third of patients with severe sepsis are immunosuppressed suggests that novel immune therapeutics must be effective for members of this population to have broad clinical utility.

Our study has several limitations. The UHC database contains more hospital diagnosis codes (up to 99) than the database used by Angus and colleagues For this reason, the Angus definition may be more sensitive and less specific at identifying cases of severe sepsis when applied to the UHC database. In addition, the academic medical centers and affiliated hospitals in the UHC database may have a greater percentage of immunosuppressed patients than other nonacademic hospitals in the United States. The positive and negative predictive values of our search strategy may vary depending on the types of hospitals that compose the database.

Recently, the ICD-9 codes that we used in this study to identify patients with severe sepsis and immunosuppressive conditions have been updated. Going forward, our strategy will require adaptation to the newer ICD-10 coding system. Another limitation is that there is no universal definition of clinical immunosuppression. Our definition was based a priori on previously reported classification schemes (2, 3, 7).

Our search strategy cannot be used in databases that do not include medication administration data. However, attempts to identify immunosuppressed patients based on medical conditions alone will result in many false positives. We recommend that this search strategy be investigated using other large databases that include medication data. Finally, we validated our search strategy through review of medical records at a single medical center. However, our data suggest that the numbers and types of immunosuppressed patients at the University of Chicago are similar to those at other academic medical centers.

In conclusion, we report on a novel strategy to identify immunocompromised patients in a large administrative database. This patient population represents a large subgroup of patients with severe sepsis and likely influences trends in incidence and mortality overall. A greater awareness of the burden of immunosuppressive medical conditions among patients with sepsis should lead investigators to examine the immune response to infection specifically for members of this population.