Neural Net Modeling of Checkpoint Inhibitor Related Myocarditis and Steroid Response

Abstract

Background

Serious but rare side effects associated with immunotherapy pose a difficult problem for regulators and practitioners. Immune checkpoint inhibitors (ICIs) have come into widespread use in oncology in recent years and are associated with rare cardiotoxicity, including potentially fatal myocarditis. To date, no comprehensive model of myocarditis progression and outcomes integrating time-series based laboratory and clinical signals has been constructed. In this paper, we describe a time-series neural net (NN) model of ICI-related myocarditis derived using supervised machine learning.

Methods

We extracted and modeled data from electronic medical records of ICI-treated patients who had an elevation in their troponin. All data collection was performed using an electronic case report form, with approximately 300 variables collected on as many occasions as available, yielding 6000 data elements per patient over their clinical course. Key variables were scored 0–5 and sequential assessments were used to construct the model. The NN model was developed in MatLab and applied to analyze the time course and outcomes of treatments.

Results

We identified 23 patients who had troponin elevations related to their ICI therapy, 15 of whom had ICI-related myocarditis, while the remaining 8 patients on ICIs had other causes for troponin elevation, such as myocardial infarction. Our model showed that troponin was the most predictive biomarker of myocarditis, in line with prior studies. Our model also identified early and aggressive use of steroid treatment as a major determinant of survival for cases of grade 3 or 4 ICI-related myocarditis.

Conclusion

Our study shows that a supervised learning NN can be used to model rare events such as ICI-related myocarditis and thus provide clinical insight into drivers of progression and treatment outcomes. These findings direct attention to early detection biomarkers and clinical symptoms as the best means of implementing early and potentially life-saving steroid treatment.

Introduction

Immune checkpoint-inhibitor (ICI)-associated myocarditis is a rare but serious side effect of treatment with ICI, with a fatality rate of around 50%.1 Clinical trials failed to detect this low frequency adverse event and almost all detailed laboratory and clinical information comes from cases reported during clinical use. Over 150 cases of ICI-related myocarditis have been reported in the post-marking period for the PD-1 inhibitors. There are cases for all of the widely used ICIs, including pembrolizumab2–11 nivolumab,12–31 the CTLA-4 inhibitor ipilimumab, either alone or in combination with nivolumab,14,16,18,31–34 and the PD-L1 inhibitors atezolizumab35 and durvalumab.36,37 A multicenter observational cohort study estimated the prevalence of ICI-related myocarditis to be approximately 1.14%.1 Additionally, numerous cases also appear under related headings such as cardiotoxicity, cardiac toxicity, myositis, and immune-related cardiovascular events. A recent retrospective study estimated the absolute 1-year risk of all cardiotoxicities including arrhythmias, pericarditis, myocarditis and heart failure to be 9.7%. Collectively, these cases have prompted guidelines for diagnosis and management.38,39

Patients with the more severe manifestations of ICI-related myocarditis have varying degrees of heart block upon EKG, reduced ejection fraction, elevated plasma troponin, CPK, AST and LDH. Endomyocardial biopsy and autopsy studies reveal massive infiltration of CD8+ lymphocytes in the hearts of most of these severe and fatal events.40 Most patients also present with concomitant immune system related adverse events (irAEs) in addition to myocarditis, such as fatigue, rash, endocrine disorders, hepatitis,41 neurological disorders,42 colitis,43 and pneumonitis.44 In a previous study in 403 patients hospitalized between 2003 and 2013, where the goal was to establish risk factors for mortality from known or suspected viral myocarditis, the main predictors of mortality were patient age >50 years, creatinine clearance <60 mL/min, ventricular tachycardia, NYHA classification >3, and elevated troponin.45

Although correlations to ICI-related myocarditis and myocarditis generally have been identified, no report has attempted a clinical disease model integrating time-related changes in laboratory findings, clinical histories, and treatment responses with clinical outcomes. One barrier to the development of such a clinical algorithm is the paucity of cases available for analysis due to the low frequency of ICI-related myocarditis. A second challenge is the relatively little data collected on each case, and lack of time-related clinical signals in particular. Machine learning methods conventionally require large datasets to make correlations between variables and an outcome of interest, but thus far have not been applied to time and magnitude in individuals. In most cases, these artificial intelligence algorithms source case report forms or adverse reaction reports, and thus rely on sparse data collected at indistinct time points designated by events, such as initiation of a drug therapy or beginning of an adverse event.

An alternative machine learning approach applies Neural Network (NN) Models to the patient medical records of target cases that are enriched for the event in question, such as we previously used for understanding bleeding events from warfarin.46 Having access to the medical records of well-defined irAE cases from a survey paper recently published by our clinical group47 we then applied the NN modeling approach to better understand time and magnitude and treatment response in ICI-related myocarditis. This model leverages known clinical covariates in a structured scoring system in order to boost model performance, generalizability, and interpretability.48 Furthermore, this approach allows us to integrate patient-level information regarding the presentation of ICI-related myocarditis such as the co-occurrence of other irAEs which may be infrequent but are highly correlated with ICI-related myocarditis in an individual patient’s clinical context.

Previously, we examined 15 cases of ICI-related myocarditis retrospectively for determinants of severe disease and mortality using conventional statistical hazard modeling methods.47 In this study, we utilize the rich time-series clinical data available from patient electronic health records to construct a NN-based predictive model of determinants of ICI-related myocarditis development, severity, response to treatments and resolution.

Methods

Study Design and Modeling

We developed a NN model of a longitudinal cohort of patients treated with ICIs at Roswell Park Comprehensive Cancer center and developed elevated serum troponin concentrations of >0.06 ng/mL - upper limit of normal for the laboratory test.47 Data were collected from electronic health records and included outpatient and inpatient health-care claims, clinical monitoring, laboratory and pharmacy data. The laboratory data include all tests performed and results. Pharmacy data include all patient demographics and medication dosing.

Patient Sample

Twenty-three cases were identified based on elevated troponin levels as demonstrating likely ICI-cardiotoxicity as determined by clinical assessment using Naranjo Score. Eleven of these patients had severe myocarditis, 4 had mild myocarditis, and in the remaining 8, the elevated troponin was ascribed to other causes such as acute coronary syndromes or exacerbations of heart failure. All patients had troponins and ECGs available before and after ICI initiation, as well as after cardiotoxicity development. Ten patients in the severe myocarditis and 3 patients in the subclinical myocarditis cohorts had echocardiograms with estimated ejection fractions. Gadolinium enhanced cardiac MRI (CMR) was performed in 3 patients with severe myocarditis. Ischemic evaluation with coronary angiography was performed in 3 patients with severe myocarditis.

Neural Net Model Inputs, Outputs, Biomarkers

Myocarditis severity was used as the primary output parameter since the goal was to define the drivers of myocarditis onset and severity. Myocarditis was expressed as a score 0–5, using criteria adapted from CTCAE guidelines (Supplementary Table S1, see Online Supplement for detailed scoring criteria). Troponin levels were also modeled as an output. We also modeled composite parameters chosen to elucidate the dependent relationships, including tumor diameter as a measure of tumor burden, combination irAEs, combination lab signals, and combination myocarditis signs and symptoms metric. See Online Supplement for more information on inputs and modeling methods.

We created several combination signals using various normalized factors listed in Table 1. Each included factor was weighted on clinical importance and the individual weighted components were averaged (if present) into the combination signals. If specific parameters were not available for a patient, they were omitted so that the average would not be affected.

Table 1.

Baseline Clinical, and Laboratory Characteristics of 23 Patients Prior to ICI Administration

	All Cases	ICI-Related Myocarditis	Other Cardiotoxicity	P-value
	n=23	n=15	n=8
Age, years	72 (68, 79)	72 (68, 80)	72 (68, 75)	0.49
Female gender	9 (39)	4 (27)	5 (63)	0.18
CV Risk Factors or Conditions
Current or prior smoking	6 (26)	5 (33)	1 (13)	0.37
Hypertension	17 (74)	12 (80)	5 (63)	0.62
Diabetes mellitus	7 (30)	4 (27)	3 (38)	0.66
CAD other than MI	5 (22)	2 (13)	3 (38)	0.29
Atrial Fibrillation	6 (26)	5 (33)	1 (13)	0.37
Checkpoint Inhibitor				0.71
Nivolumab	7 (30)	4 (27)	3 (38)
Pembrolizumab	11 (48)	7 (47)	4 (50)
Atezolizumab	2 (8.7)	1 (6.7)	1 (13)
Nivolumab & Ipilimumab	3 (13)	3 (20)	0
Echocardiography
LVEF %	63 (49, 70)	62 (35, 65)	63 (55, 70)	0.49
ECG
QRS interval, ms	97 (86, 116)	102 (86, 129)	91 (90, 98)	0.73
QTc interval, ms	439 (409, 468)	441 (423, 473)	412 (401, 444)	0.36
Cardiac Biomarkers
Troponin T, ng/mL	0.52 (0.12, 2.85)	0.66 (0.12, 3.82)	0.46 (0.11, 1.36)	0.50
CPK, IU/L	71 (32, 680)	122 (37, 964)	41 (24, 51)	0.052
CPK-MB, IU/L	3.7 (1.8, 61)	7.5 (2.1, 89)	1.9 (1.8, 4.9)	0.21
BNP, pg/mL	423 (108, 664)	423 (108, 664)	-
Laboratory Measurements
AST, U/L	95 (43, 173)	129 (47, 204)	43 (22, 43)	0.08
ALT, U/L	45 (32, 168)	85 (33, 233)	32 (31, 45)	0.27
LDH, U/L	541 (419, 742)	672 (394, 1194)	478 (423, 541)	0.44
White blood cells, 109/L	8.8 (7.1, 14)	9.9 (7.3, 14.8)	7.8 (6.9, 8.8)	0.59
Lymphocytes 109/L	871 (736, 1457)	871 (693, 1608)	1043 (780, 1307)	1.0
Neutrophils 109/L	7194 (5156, 12,381)	8035 (5294, 13,542)	5654 (4609, 6699)	0.33

Notes: Values are presented as median (IQR) or n (%). Continuous data are analyzed using Wilcoxon rank sum test and categorical data are analyzed using Chi-square or Fisher’s exact test, as appropriate.

Abbreviations: CV, cardiovascular; CAD, coronary artery disease; MI, myocardial infarction; LVEF, left ventricular ejection fraction; CPK, creatine phosphokinase; CPK-MB, creatine kinase-MB; BNP, B-type natriuretic peptide; AST, aspartate aminotransferase; ALT, alanine aminotransferase; LDH, lactate dehydrogenase.

The following combination signals and constituents were developed for entry into the model:

• Combination Adverse Event Signal (“CombAdvEvt”): myocarditis, pruritis, myalgia, dyspnea, chest pain, bradycardia, edema, diplopia, ptosis, pancreatitis, thyroiditis, hepatitis, colitis, myocardial infarction, optic nerve disorders, myasthenia gravis, rash, encephalitis, increased creatinine or BUN, fatigue, diarrhea, pneumonitis, and polyneuropathy.

• Combination Myocarditis Metric (“CombLoadMyoMetric”): respiratory rate, body temperature, QRS, troponin (not normalized), use of antiPD1, use of antiCTLA4, and WBC.

• “Organ Metric”: total bilirubin, serum creatinine, oxygenation, platelets, hypotension, and SOFA Score.

• Combination Lab Signal (“CombLabSignal”): troponin Score, BNP Score, AST Score, LDH Score, QRS Score, Lactate Score, CPK Score and CPK-MB Score.

Neural Net Modeling

Modeling for the effects of ICIs was done in MatLab by loading all patient data and their corresponding values as part of a structured time-based array. As the data were loaded, input data were verified using the unique patient identifiers and stored in a sequential manner based on their time stamps. Patients were assigned identification numbers so that duplicate entries do not occur. Minimum mean square error (MMSE) was calculated to rank order the model outputs. For detailed description of the neural net model structure, see the Online Supplement.

Results

Sample Characteristics

We identified 23 cases of troponin elevation during ICI therapy as defined by troponin >0.06 ng/mL. Of these 23, 15 had ICI-related myocarditis as defined by the criteria, while the remaining 8 cases had other cardiotoxicities during ICI treatment. All available data was collected on each case. In the final model, there were 140,916 data elements collected in the 23 study patients. While this averages to 6126 data points per patient, the Electronic Health Record derived datasets were more concentrated in the severe cases who were hospitalized and survived longer. Table 1 shows little difference at baseline in clinical or demographic characteristics upon comparison of the 15 ICI-related myocarditis and 8 other cardiotoxicity cases. The patients were of similar age and underlying disease profile, received similar treatments, and in most cases had similar clinical monitoring. Patients who presented with ICI-related myocarditis had similar clinical laboratory biomarkers to patients who presented with other cardiotoxicities. CPK and AST trended higher in patients who had ICI-related myocarditis, although these associations were not statistically significant.

The baseline characteristics of the 15 cases of ICI associated myocarditis are presented in Table 2. There were 11 patients who reached a score of 3 or 4, values that are associated with hospital care and in the case of score of 4, ICU care. These patients demonstrated multi-organ involvement by multiple lab tests of organ function, and usually more than one other immune-related adverse event. In general, the myocarditis score was a marker of systemic involvement in the inflammatory process as evidenced by the elevation in LFTs. Specifically, LDH was markedly elevated at baseline in patients who died of ICI-related myocarditis in comparison to survivors and positive controls in the same population who experienced non-ICI-related troponin elevations (such as NSTEMI, heart failure exacerbations) (Table 2). As shown in Table 2, there were no other major differences between baseline pre-ICI therapy conditions and biomarkers at presentation in the 4 cases of fatal ICI-related myocarditis vs the 11 patients with myocarditis who survived.

Table 2.

Baseline Clinical and Laboratory Characteristics Between Patients with Fatal and Non-Fatal ICI-Related Myocarditis, vs Other Cardiac Conditions on ICI Treatment

	Non-Fatal ICI-Related Myocarditis	Fatal ICI-Related Myocarditis and Died	Other Cardiotoxicity
	n=11	n=4	n=8
Age, years	72 (68, 80)	74 (58, 81)	72 (68, 75)
Female gender	3 (27)	1 (25)	5 (63)
CV Risk Factors or Conditions
Current or prior smoking	4 (36)	1 (25)	1 (13)
Hypertension	9 (82)	3 (75)	5 (63)
Diabetes mellitus	3 (27)	1 (25)	3 (38)
CAD other than MI	1 (9)	1 (25)	3 (38)
Atrial Fibrillation	3 (27)	2 (50)	1 (13)
Checkpoint Inhibitor
Nivolumab	2 (18)	2 (50)	3 (38)
Pembrolizumab	5 (45)	2 (50)	4 (50)
Atezolizumab	1 (9)	-	1 (13)
Nivolumab & Ipilimumab	3 (27)	-	-
Echocardiography
LVEF %	60 (25, 65)	65 (49, 81)	63 (55, 70)
ECG
QRS interval, ms	99 (86, 136)	109 (80, 112)	91 (90, 98)
QTc interval, ms	438 (414, 492)	453 (424, 473)	412 (401, 444)
Cardiac Biomarkers
Troponin T, ng/mL	1.52 (0.33, 5.78)	0.15 (0.10, 0.28)	0.46 (0.11, 1.36)
CPK, IU/L	466 (76, 1445)	54 (35, 376)	41 (24, 51)
CPK-MB, IU/L	68 (6.9, 138)	1.8 (1.15, 3.1)	1.9 (1.8, 4.9)
BNP, pg/mL	423 (108, 664)	-	-
Laboratory Measurements
AST, U/L	146 (47, 454)	95 (34, 204)	43 (22, 43)
ALT, U/L	168 (28, 235)	57 (33, 112)	32 (31, 45)
LDH, U/L	533 (338, 757)	4258 (727, 7789)	478 (423, 540)
White blood cells, 109/L	8.9 (7.3, 14)	11 (6.5, 16)	7.8 (6.9, 8.8)
Lymphocytes 109/L	871 (502, 1619)	863 (693, 1033)	1043 (780, 1307)
Neutrophils 109/L	8035 (5640, 12,381)	9507 (5294, 13,721)	5654 (4609, 6699)

Note: Values are presented as median (IQR) or n (%).

Abbreviations: CAD, coronary artery disease; MI, myocardial infarction; LVEF, left ventricular ejection fraction; CPK, creatine phosphokinase; CPK-MB, creatine kinase-MB; BNP, B-type natriuretic peptide; AST, aspartate aminotransferase; ALT, alanine aminotransferase; LDH, lactate dehydrogenase.

Time Course of Clinical Progression

Four of the 15 patients who developed ICI-related myocarditis had fulminant myocarditis resulting in death. The time course of clinical progression of the 11 non-fatal cases and 4 fatal cases of ICI-related myocarditis are shown in Figures 1 and 2, respectively.

Figure 1.

Biomarker time course in 11 patients with ICI myocarditis and lived. Time course of selected biomarkers in 11 patients with elevated troponin caused by ICI associated myocarditis. Time zero is onset of myocarditis, shown by change in myocarditis severity score from zero. Myocarditis was assigned a clinical severity score from 0–5 with 1 corresponding to symptomatic disease, 2 to symptomatic disease + abnormal biomarkers, 3 to myocarditis-related hospitalization, 4 to ICU care, and 5 assigned at the time of death. All 11 patients survived the ICI myocarditis episode displayed in this figure, although a score of 4 was reached by patient 1, 4, 8 and 9. Notable is the close association between prednisone equivalent dose score and the decline of myocarditis score as well as both laboratory and clinical indices of myocarditis. Among the important biomarkers that track with use of steroids, the QRS rose before prednisone score and declined rapidly afterwards. Lymphocyte count declined with myocarditis severity score and rose after it resolved.

Figure 2.

Biomarker time course in 4 patients with fatal ICI myocarditis. Time course of selected biomarkers in 4 patients with elevated troponin caused by ICI-related myocarditis. Time zero is onset of myocarditis, shown by change in myocarditis severity score from zero. Myocarditis was assigned a clinical severity score from 0–5 with 1 corresponding to symptomatic disease, 2 to symptomatic disease + abnormal biomarkers, 3 to myocarditis-related hospitalization, 4 to ICU care, and 5 assigned at the time of death. These 4 patients died primarily as a result of ICI myocarditis or immediate sequelae. Notable is the close association between steroid use and dose (represented in prednisone equivalents by “PrednisoneEq_Score”) and the decline of myocarditis score as well as both laboratory and clinical indices of myocarditis. However, in most cases prednisone dose was too low and started too late to be lifesaving. QRS rose before prednisone score but did not decline with inadequate prednisone dose. Lymphocyte count declined with myocarditis and did not recover with inadequate prednisone dose.

As shown in Figures 1 and 2, lymphocyte count was an important predictor of the onset severity and duration of ICI-related myocarditis. Greater decline in lymphocyte counts preceded severe cases of ICI-related myocarditis, and lower lymphocyte counts were correlated with the development of severe ICI-related myocarditis. After high dose steroids, the lymphocyte counts rose in most of these severe cases. Cardiac imaging exhibited inflammatory changes such as cardiac edema and increased wall thickness, as well as onset of heart failure in settings where there was either heart block or extensive myocardial damage from inflammation.

In Figure 1, the individual time plots of biomarkers are shown for 11 patients with ICI-related myocarditis who survived. In these data plots, time zero is onset of myocarditis, shown by change in myocarditis score. Notable is the close association between steroid use and the decline of myocarditis score as well normalization of both laboratory and clinical indices of myocarditis. Treatment with sufficient doses of steroids was associated with resolution of QRS widening and reduced risk of death. Early and adequate steroid use is lifesaving in these cases. Consistent with the effects of myocarditis on lymphocyte trafficking, the lymphocyte count declined with myocarditis and rose after it resolved from use of high dose corticosteroids.

Figure 2 shows the time course of NN model-selected clinical covariates in 4 patients with fatal outcome (myocarditis score 5) caused by ICI-related myocarditis. These 4 patients died primarily because of ICI-myocarditis or immediate sequelae. In these cases, there was insufficient decrease in myocarditis score following the use of immunosuppressive therapy, which often was initiated later in the time course of their clinical progression.

The time course of clinical progression of the 8 patients who experienced troponin elevations for other causes while on ICI is shown in Figure 3. Parameters chosen for display are highly correlated with ICI-related myocarditis; patients without ICI-related cardiotoxicities therefore show little change in these biomarkers over time. Although three of these cases did receive corticosteroids due to initial suspicion of myocarditis, there was seldom any evident change in the biomarkers of myocarditis, identifying these cases as not having an inflammatory cause in the heart. As shown in Figure 3, some of these cases were so similar upon clinical presentation, that they were treated with steroids. In each of these, the steroids were stopped after the actual cause was determined, and it was concluded that ICIs were not the cause of their cardiotoxicity. Further discussion of the precise clinical diagnostics used to define myocarditis cases is found in our clinical paper (47).

Figure 3.

Biomarker time course in 8 ICI treated patients with other cardiotoxicity. Time course of selected biomarkers in 8 patients with elevated troponin that was determined not to be caused by ICI associated myocarditis. Two patients had a rise in troponin associated with worsening of CHF, 5 patients had coronary artery events primarily ischemic, and one patient had tumor hyper-progression with obstructing cardiopulmonary disease. Time zero here is the onset of troponin rise, shown by the change in troponin score. Although 3 of these cases did receive prednisone, there was seldom any evident change in the biomarkers of myocarditis, identifying these cases as not having an inflammatory cause in the heart.

Neural Net Modeling and MMSE Analyses

An MMSE analysis was performed for predictors of ICI-related myocarditis by comparing the clinical predictors of myocarditis score and troponin elevation between patients who developed ICI-myocarditis and those who developed troponin elevations of other cause (Table 3). The patients who developed troponin elevations of other causes were used as a positive control to analyze the determinants of ICI-related myocarditis vs other cardiac presentations in the same patient population. The top predictors of clinical myocarditis severity score in patients with confirmed ICI-related myocarditis were the combined myocarditis metric (“CombLoadMyoMetric”), the combined lab signal (“CombLabSignal”), CPK-MB, ALT, troponin and QRS interval. The top predictors of troponin in positive control patients who did not have myocarditis were myocarditis metric (“CombLoadMyoMetric”), the combined lab signal (“CombLabSignal”) and lactate. The modeling outputs presented some similar trends to the clinical time progression presented by the ICI-related myocarditis patients (Table 3). EKG changes were largely non-specific for ICI-related myocarditis, as they were also closely associated with the presentation of other cardiac conditions but were very predictive of myocarditis score in patients who did have ICI-related myocarditis (Table 3). Markers of organ failure and life-threatening conditions, such as shock, use of vasopressors, blood pressure and oxygenation were only found in the most severe subset, so while they were not as strongly associated with clinical outcomes when examining all 15 cases, they did strongly associate with clinical outcomes when examining the differences between the 11 non-fatal vs the 4 fatal cases (as shown in Table 4). The biomarker comparison between the positive controls and the 15 cases of ICI-related myocarditis reveal how similar non-ICI related cardiac presentations in ICI treated patients are to ICI-related myocarditis (Table 3). Once the diagnostics are performed, most patients differentiate from a lack of inflammatory involvement in the heart, different conduction defects, and the cardiotoxicity does not improve with steroid therapy, and may worsen if the cause is myocardial infarction.

Table 3.

All 23 Patients with Elevated Troponin: Comparison of Troponin and Myocarditis Score in 15 Patients with ICI-Related Myocarditis vs 8 Patients with Other Cardiotoxicity Causes for Elevated Troponin

Patient Group	All Patients Given ICIs with Elevated Troponin				Patients with Troponin Elevation from ICI Myocarditis				Patients with Troponin Elevation Not Related to ICI-Myocarditis
# Cases in Model Run	23		23		15		15		8		8
Output for	Troponin		Myocarditis Score		Troponin		Myocarditis Score		Troponin		Comb LabSignal
Mean Prediction Error per patient	6.73%		7.00%		9.30%		10.10%		7.20%		4.90%
% Improvement over Null	13.40%		11.90%		15%		16.20%		17.40%		21.80%
	Input	Error	Input	Error	Input	Error	Input	Error	Input	Error	Input	Error
Ordered Output	All Kept	0.0%	All Kept	0.0%	Troponin I	0.0%	Myocarditis	0.0%	Troponin I	0.0%	All Kept	0.0%
	Troponin I	0.8%	ADV EVT Myocarditis	0.3%	All Kept	0.3%	All Kept	4.4%	All Kept	1.4%	Comb LabSignal	0.6%
	Troponin Score	1.6%	CPK MB	16.1%	Troponin Score	1.5%	Comb Load MyoMetric	11.1%	Troponin Score	6.1%	Troponin I	2.2%
	CombLoad MyoMetric	9.8%	CPKMB Score	16.3%	CombLoad MyoMetric	8.7%	CPKMB Score	17.2%	CombLoad MyoMetric	10.6%	Troponin Score	2.4%
	Comb Lab Signal	15.3%	Comb Load MyoMetric	17.3%	Comb LabSignal	10.6%	ALT Score	18.4%	Lactate	16.6%	CombLoad MyoMetric	2.5%
	ADV EVT Myocarditis	19.0%	Comb ADV EVT	18.0%	Myocarditis	11.5%	Troponin Score	18.8%	Lactate Score	16.8%	LDH Score	2.9%
	ALT	20.5%	Comb Lab Signal	20.8%	ALT	15.0%	Comb LabSignal	18.9%	Comb LabSignal	19.5%	Lactate Score	4.5%
	Lactate Score	21.6%	ALT	23.8%	CPKMB Score	17.7%	Troponin I	19.1%	AST Score	26.1%	LDH	4.7%
	BNP	22.5%	ALT Score	24.1%	BNP	18.9%	QRS Score	20.9%	SpO2	26.7%	ALT	4.7%
	QRS	23.2%	Troponin I	24.3%	QRS Score	19.0%	Blood gas pH	25.4%			QRS	5.0%
	CPKMB Score	24.1%	Troponin Score	24.5%	Lactate Score	21.0%	Lactate Score	25.6%	QRS Score	27.2%	QRS Score	5.4%
	CombADV EVT	24.3%	ADV EVT Complete heart block	27.4%	AST Score	21.4%	Complete heart block	27.2%	HCT	27.8%	AST Score	5.5%
	AST Score	24.6%	Immunosuppressant	29.1%	Calcium	23.4%	AST Score	27.4%	LDH Score	28.1%	ActIndex	5.6%
	Calcium	27.0%	QRS Score	29.1%	BPdiast	23.4%	Calcium	27.6%	ALT	28.6%	BPdiast	6.2%
	BPdiast	27.1%	Diag Abnormal ECG	29.6%	LDH	25.5%	BNP Score	29.5%	AST	28.7%	ADV EVT Fatigue	7.8%
	LDH	28.0%	QRS	29.7%	CPK Score	25.9%	BPdiast	30.3%	HeartRate	28.8%	BNP Score	8.0%
	HCT	29.7%	BNP Score	31.1%	LDH Score	26.1%	Immunosuppressant	30.3%	CPK Score	28.9%	ADV EVT Dyspnea	8.1%
	WBC Score	30.2%	Lactate	31.4%	OrganMetric	27.6%	WBC Score	30.6%	Neutrophils-Band	29.1%	INR	8.7%
	ADV EVT Dyspnea	30.2%	Antibiotic	31.6%	HCT	27.7%	CPK	31.5%	BNP Score	29.3%	HCT	8.9%
	SpO2	30.7%	AST Score	31.7%			Antibiotic	31.7%	BNP	29.8%	CPKMB Score	9.1%
	CPK Score	30.9%	Lactate Score	31.9%	PDL1 exp	28.2%	HCT	31.9%	Drug dose prednisone	30.6%	Drug equiv prednisone	9.1%
	PDL1 exp	30.9%	HCO3	32.2%	INR	28.3%	Myoglobin	32.0%	Drug equiv prednisone	30.6%	SpO2	9.2%
	INR	31.1%	Calcium	32.5%	Chills	28.4%			WBC Score	30.8%	ADV EVT Chest pain	9.6%
	Platelets	31.2%	BPdiast	32.7%	WBC Score	28.5%	Myocardial infarction	34.1%	Alk Phos	31.3%	Drug dose prednisone	9.6%
	OrganMetric	31.4%	Loop diuretic	33.2%	Myocardial infarction	28.7%	Antiviral	34.2%	BPdiast	31.4%	ADV EVT Myocardial infarct	9.6%
	FIO2	31.6%	CPK	33.3%	PR interval	28.8%	Edema limbs	34.6%	aPTT	31.4%	OrganMetric	9.7%
	ADV EVT Pruritus	31.6%			FIO2	28.9%	OrganMetric	34.9%	Albumin	31.5%	Alk Phos	9.7%
	ADV EVT Edema	31.8%			Pruritus	28.9%	Dyspnea	35.3%	Calcium	32.1%	PO2	9.7%
	BPsys	31.8%			Sodium	28.9%	QRS	35.3%	TempF	32.5%	Prednisone Eq Score	9.7%
	Magnesium	31.8%			Dyspnea	29.0%	LDH Score	35.5%	ADV EVT Fatigue	32.5%	ADV EVT Edema	9.7%
	Sodium	31.8%			Edema in limbs	29.0%	PDL1 exp	35.5%	Magnesium	33.0%	QT interval	9.8%
	Drug dose prednisone	31.9%			Corticosteroid	29.1%	Myalgia	35.6%	ADV EVT Dyspnea	33.0%	ESR	9.9%
	HeartRate	32.0%			Platelets	29.2%	Drug dose prednisone	35.7%	CPKMB Score	33.0%	Ferritin	9.9%
	Drug equiv prednisone	32.0%			MPV	29.2%	Drug equiv prednisone	35.7%	Tumor Score	33.1%	PDL1 exp	9.9%
							Rhabdomyolysis	35.8%	ADV EVT Chest pain	33.1%

Abbreviations: CombLoad Myometric, combination myocarditis metric; Comb LabSignal, combination laboratory signal; ADV EVT, adverse event; CombADV EVT, combination adverse event signal; CPK, creatine phosphokinase; CPK-MB, creatine kinase-MB; BNP, B-type natriuretic peptide; AST, aspartate aminotransferase; ALT, alanine aminotransferase; LDH, lactate dehydrogenase; Alk Phos, alkaline phosphate; BPdiast, diastolic blood pressure; BPsys, systolic blood pressure; HCT, hematocrit; WBC, white blood cells; PDL1, Programmed death-ligand 1; SpO2, peripheral O2 saturation; FIO2, fraction of inspired oxygen; PO2, partial pressure of oxygen; HCO3, bicarbonate; MPV, mean platelet volume; aPTT, activated partial thromboplastin time; INR, international normalized ratio.

Table 4.

Model MMSE Outputs of 15 ICI-Related Myocarditis Cases, Comparing 11 Non-Fatal and 4 Fatal Cases

Patient Group	Non-Fatal ICI-Related Myocarditis				Fatal ICI-Related Myocarditis
# Cases in Model Run	11		11		4		4
Output for	Myocarditis score		Troponin		Myocarditis Score		Troponin
Mean Prediction Error per patient	6%		8.30%		21.50%		14.50%
% Improvement over Null	28.20%		37.50%		44.80%		24.20%
	Input	Error	Input	Error	Input	Error	Input	Error
Ordered Output	Myocarditis Score	0.0%	Troponin I	0.0%	All Kept	0.0%	Troponin I	0.0%
	All Kept	3.6%	Troponin Score	0.2%	CPK MB	0.2%	Troponin Score	0.7%
	Comb Load Myometric	18.1%	All Kept	2.1%	CPKMB Score	0.5%	CPKMB Score	5.6%
	Comb ADV EVT	19.3%	Comb Load Myometric	8.6%	ALT Score	0.7%	All Kept	5.6%
	Calcium	23.9%	Lactate Score	15.4%	Troponin Score	1.4%	CPK MB	5.9%
	Troponin I	24.8%	Comb LabSignal	15.5%	QRS	2.0%	Myocarditis	6.2%
	Troponin Score	26.0%	Myocarditis Score	19.5%	Lactate Score	3.2%	ALT Score	6.3%
	Comb LabSignal	28.6%	Calcium	19.9%	QRS Score	3.2%	QRS Score	7.1%
	Lactate	28.8%	HCT	20.1%	Comb Load Myometric	4.3%	Comb ADV EVT	7.2%
	HCT	29.9%	ALT Score	21.1%	Comb LabSignal	5.6%	Comb Load Myometric	7.3%
	ALT Score	31.0%	BNP Score	21.3%	AST	6.1%	Comb LabSignal	8.2%
	BNP Score	31.0%	OrganMetric	21.7%	Myocarditis Score	6.2%	AST	10.2%
	BPdiast	33.1%	LDH	22.6%	AST Score	6.9%	Lactate Score	13.5%
	Magnesium	33.2%	BNP	23.1%	CO2	11.6%	AST Score	14.5%
			Platelets	23.5%	Comb ADV EVT	12.4%	BNP Score	16.4%
	PO2	34.0%	AST Score	24.8%			OrganMetric	20.7%
	Drug dose prednisone	34.0%	Comb ADV EVT	25.1%	ADV EVT 3rd heart block	16.8%	CPK Score	20.9%
	QT interval	34.1%	ADV EVT Chills	25.4%	Proton pump inhibitor	16.9%
	AST Score	34.3%	BPdiast	25.7%	OrganMetric	17.5%	LDH Score	25.5%
	Prednisone Dose	34.4%	QRS	25.8%	HCT	17.7%	Sodium	26.5%
	QRS	34.4%	PT interval	26.1%	BNP Score	18.0%	BPdiast	26.6%
	Platelets	34.5%	HeartRate	26.3%	Calcium	18.1%	WBC Score	26.7%
	QRS Score	34.6%	ADV EVT Hypothyroidism	26.3%	BPdiast	18.7%	Calcium	28.4%
	SpO2	34.6%	WBC Score	26.4%	LDH	18.8%	HCT	28.8%
	ALT	34.8%	Immunosuppressant	26.4%	LDH Score	18.8%	Immunosuppressant	29.7%
	ADV EVT Myocardial inf	35.0%	TSH	26.4%	BNP	19.3%	BUN	29.9%
	OrganMetric	35.0%	ADV EVT Diarrhea	26.4%	WBC Score	20.1%	MAP	30.1%
	ADV EVT Sepsis	35.3%	CPK	26.5%	Blood gas pH	21.1%	ADV EVT Dyspnea	30.1%
	WBC Score	35.3%	Glucose	26.6%	Antibiotic	21.4%	Neutrophils	30.3%
	O2S	35.5%	Drug dose prednisone	26.7%	PCO2	22.9%	PR interval	30.4%
	CPKMB Score	35.5%	ADV EVT Hypotension	26.7%	Phosphate	23.2%	Corticosteroid	30.4%
	CPK MB	35.7%	BUN	26.7%	Immunosuppressant	23.4%	Eosinophils	30.6%
	ADV EVT Rash maculopapular	35.8%			ADV EVT Dyspnea	23.9%	RBC	30.6%
	Albumin	35.8%	Prednisone Dose	26.9%	CPK Score	23.9%	RespRate	30.7%
	RBC	36.1%	Albumin	26.9%	Corticosteroid	24.1%	PO2	30.7%
	FIO2	36.2%	Ferritin	26.9%	HCO3	24.8%	ADV EVT Chills	30.8%
			PrednisoneEq Score	26.9%	Insulin	25.0%	Magnesium	30.9%

Abbreviations: CombLoad Myometric, combination myocarditis metric; Comb LabSignal, combination laboratory signal; ADV EVT, adverse event; CombADV EVT, combination adverse event signal; CPK, creatine phosphokinase; CPK-MB, creatine kinase-MB; BNP, B-type natriuretic peptide; AST, aspartate aminotransferase; ALT, alanine aminotransferase; LDH, lactate dehydrogenase; Alk Phos, alkaline phosphate; BPdiast, diastolic blood pressure; BPsys, systolic blood pressure; HCT, hematocrit; WBC, white blood cells; RBC, red blood cells; SpO2, peripheral O2 saturation; FIO2, fraction of inspired oxygen; PO2, partial pressure of oxygen; HCO3, bicarbonate; TSH, Thyroid-stimulating hormone; BUN, blood urea nitrogen; prednisone eq score, scored steroid dose in mg of prednisone.

Analysis of 4 Cases of Fatal ICI-Related Myocarditis

A separate MMSE analysis was performed for predictors of the output of ICI-related myocarditis fatality by investigating the association between clinical myocarditis score (and troponin) in patients who survived and in patients who died. All patients died of ICI-related myocarditis within 30 days of the onset of symptoms.

Model errors represent the strength of the relationship between the input and the output, ie, the lower the model error, the stronger the relationship between the predictor and the outcome. For example, changes in the time and magnitude of CPK-MB are associated with a 0.2% model error in patients with fatal myocarditis, indicating that only 0.2% of CPK-MB data points would be expected to not correspond to the biomarker profile of fatal ICI-related myocarditis. Our results showed that CPK-MB, troponin score, QRS interval and ALT score were the most closely correlated inputs with the outcome of a fatal myocarditis score of 5 (Table 4). These inputs were associated with much lower model errors for the outcome of myocarditis score when myocarditis was fatal compared to when it was not, indicating that they were both highly predictive and specific to the outcome of severe myocarditis. For example, CPK-MB was associated with a 0.2% error in the patients with fatal myocarditis whereas it was associated with a 35.7% error in patients with non-fatal myocarditis. Interestingly, ALT score was associated with a 0.7% error for patients with fatal myocarditis whereas it was associated with 31% error for non-fatal myocarditis. Troponin score was associated with a 1.4% error in fatal myocarditis whereas it was associated with a 26% error in non-fatal myocarditis. QRS interval was associated with a 2% error in fatal myocarditis whereas it was associated with a 34.4% error in non-fatal myocarditis. Physiologic organ failure markers such as hypotension, lactate, and presence of third-degree heart block were also associated with the development of a fatal myocarditis score.

Steroid Effect on ICI-Related Myocarditis Cases

Most patients received high dose steroids in the treatment of ICI-related myocarditis. In the clinical data, early and aggressive use of high dose corticosteroids was often the key finding in those with severe ICI-related myocarditis who survived. We examined the cases where optimal steroids resulted in a positive outcome, so we selected these 6 cases as enriched for steroid effect on myocarditis score. Figure 4 shows the myocarditis score and the prednisone biomarkers vs the time of myocarditis onset. Patients who responded received treatment on average 2-days earlier than patients who did not respond, and received higher doses earlier and longer. With rapid initiation of high dose steroids, responders typically experienced myocarditis resolving to grade 2 or less toxicity (correlating to symptomatic disease but not hospitalized), within 10–15 days of symptom onset. Patients who responded to steroids also often received prolonged courses of high dose steroids after achieving a myocarditis severity score of 3 or higher. Patients who did not respond often had steroid doses reduced while their myocarditis severity scores remained high, indicating inadequate treatment.

Figure 4.

Prednisone vs myocarditis score. The averaged prednisone equivalent dose and myocarditis score are shown for patients with a response (left) and patients who did respond to treatment (right). In patients who responded, they received treatment on average 2-days earlier than patients who did not respond, and received higher doses earlier. Patients recovered from myocarditis often required prolonged high dose steroids before myocarditis scores returned to baseline.

Leave-One-Out (LOO) Model Validation

We then tested the robustness of the MMSE rankings of these 6 patients by leave-one-out (LOO) iterations, running the NN model for each group of 5 where one case was left out (Table 5). We also analyzed the subgroup of patients who responded to prednisone compared to prednisone non-responders for the outcome of myocarditis score, in order to determine clinical predictors of clinical decompensation in patients with steroid non-response. Rankings of clinical predictors were similar between iterations where one case was left out, indicating that the model was robust. For example, the combination parameters “CombLoadMyoMetric” and “CombAdvEvt” were the top two ranking parameters in 5 out of 7 iterations of the MMSE analysis. Troponin and troponin score were also highly correlated with the outcome for myocarditis score in every iteration of the analysis. Thus, model predictability is not driven by a single subject and is representative across the steroid treated population. Additionally, we analyzed the outcome of myocarditis score for patients that were prednisone non-responders. These patients displayed a markedly altered biomarker profile, with the top predictors of myocarditis score being CPK-MB, lactate, and CPK.

Table 5.

Model MMSE Outputs for LOO Analysis of ICI-Related Myocarditis Cases, Comparing 6 Patients Who Had Clinical Resolution of Myocarditis in Response to Prednisone Treatment with 4 Non-Responder

Patients Included	All Steroid Responders		Patients 1,4,5,8,9		Patients 1,4,5,8,11		Patients 1,4,5,9,11		Patients 1,4,8,9,11		Patients 1,5,8,9,11		Patients 4,5,8,9,11		Non-Responder
# Cases in Model Run		6		5		5		5		5		5		5		4
Mean Prediction error per patient		3.7%		4.5%		4.5%		3.6%		4.2%		4.6%		4.5%		7.2%
% Improvement over Null		43.07%		39.78%		36.21%		40.02%		42.06%		37.88%		40.00%		46.43%
	Input	Error	Input	Error	Input	Error	Input	Error	Input	Error	Input	Error	Input	Error	Input	Error
	ADV EVT Myocarditis	0.0%	ADV EVT Myocarditis	0.0%	ADV EVT Myocarditis	0.0%	ADV EVT Myocarditis	0.0%	ADV EVT Myocarditis	0.0%	ADV EVT Myocarditis	0.0%	ADV EVT Myocarditis	0.0%	ADV EVT Myocarditis	0.0%
	AllKept	0.9%	AllKept	3.4%	AllKept	14.4%	AllKept	4.8%	AllKept,	2.8%	AllKept	14.6%	AllKept	3.4%	AllKept	0.1%
	CombLoad MyoMetric,	32.7%	Comb ADV EVT	28.8%	Comb ADV EVT	43.5%	CombLoad MyoMetric	40.5%	Comb ADV EVT	28.3%	CombLoad MyoMetric,	49.1%	CombLoad MyoMetric	14.2%	CPKMB Score	0.8%
	Comb ADV EVT	39.0%	CombLoad MyoMetric	30.8%	CombLoad MyoMetric	45.4%	Troponin I	40.7%	CombLoad MyoMetric,	32.9%	Comb ADV EVT	58.3%	Troponin I	24.3%	Lactate	5.8%
	Troponin Score	41.3%	Troponin Score	40.9%	Troponin I	52.0%	Troponin Score	44.9%	Lactate	45.9%	Troponin Score	65.2%	Troponin Score	24.8%	Lactate Score	6.2%
	Troponin I	45.0%	Comb LabSignal	44.3%	ADV EVT Myocardial infarction	53.0%	BNP	48.4%	Lactate Score	46.3%	Troponin I	69.7%	Lactate	29.4%	CPK	9.5%
	Comb LabSignal	49.2%	Prednisone Eq Score	47.0%	Troponin Score	53.0%	Comb ADV EVT	52.9%	AST	50.1%	Comb LabSignal	69.9%	Lactate Score	29.8%	CPK MB	11.7%
	BNP	51.9%	AST Score	50.1%	Comb LabSignal,	54.3%	Comb LabSignal,	54.8%	Troponin Score	54.2%	AST Score	71.7%	CombLab Signal	31.6%	CombLoad MyoMetric	11.7%
	AST Score	52.8%	Troponin I	51.1%	AST Score	56.2%	Prednisone Eq Score	55.7%	Troponin I	54.8%	CPK	74.8%	Comb ADV EVT	38.5%	CPK Score	12.5%
	Lactate Score	52.9%	AST	51.1%	AST	56.7%	ALT	56.7%	AST Score	56.0%	Lactate	74.9%	Prednisone Eq Score	42.3%	ALT Score	13.8%
	CPK Score	54.0%	CPK	51.4%	Macrolide	59.1%	Drug dose prednisone	58.2%	Comb LabSignal,	58.4%	CPK Score	78.4%	BNP	44.5%	Troponin Score	14.5%
	Lactate	56.0%	Lactate Score	51.4%	CPK	59.5%	AST	58.5%	ADV EVT Myocardial infarction	59.1%	ALT	78.6%	AST Score	47.3%	Comb ADV EVT	14.6%
	Prednisone Eq Score	57.4%	Lactate	52.0%	Penicillin	60.5%	Drug equiv prednisone	59.2%	Macrolide	59.6%	ADV EVT Myocardial infarction	81.3%	AST	49.2%	ALT	15.0%
	ADV EVT Myocardial infarction	57.5%	CPK Score	52.5%	Drug equiv prednisone	60.6%	CPK	59.2%	Drug dose prednisone	59.8%	Lactate Score	81.3%	BNP Score	50.3%	QRS Score	15.4%
	ALT Score	57.6%	BNP Score	52.8%	ALT	61.1%	ALT Score	59.4%	ALT	59.8%	AST	82.0%	ADV EVT Myocardial infarction	51.0%	Troponin I	15.6%
	AST	57.8%	Supplement	52.8%	CPK Score	62.2%	AST Score	59.7%	Drug equiv prednisone	60.0%	SCr	83.5%	CPK	51.9%	Comb LabSignal	22.6%
	Drug dose prednisone	58.0%	ADV EVT Myocardial infarction	53.4%	BNP	62.5%	ECHO LVEDV	60.4%	First generation antipsychotic	61.0%	Penicillin	84.3%	Drug equiv prednisone	52.3%	Prednisone Eq Score	25.3%
	Drug equiv prednisone	58.3%	Penicillin	54.5%	ALT Score	63.0%	CPK Score	61.0%	ALT Score	61.0%	Prednisone Eq Score	84.4%	Drug dose prednisone	52.8%	BNP Score	28.4%
	Macrolide	59.5%	Drug equiv prednisone	54.8%	Antiemetic	63.0%	Insulin	61.9%	H 2 antagonist,	61.0%	Macrolide	84.9%	Albumin	54.0%	BNP	31.3%
	CPK	59.5%	ALT Score	55.0%	Drug dose prednisone,	63.2%	QRS Score	61.9%	SCr	61.1%	ALT Score	85.8%	ALT Score	54.1%	AST	43.2%
	SCr	60.5%	Drug dose prednisone	55.0%	SCr	64.1%	Diag Abnormal ECG	62.1%	Prednisone Eq Score	61.2%	BNP	86.3%	ALT	54.2%	HeartRate	43.3%
	BNP Score	61.1%	Macrolide	55.3%	ADV EVT Neuropathy	64.2%	ADV EVT Dyspnea	62.2%	BNP Score	61.2%	Drug equiv prednisone	87.0%	Penicillin	54.4%	AST Score	43.9%
	Penicillin	61.2%	Tachycardia	55.6%	Electrolyte	64.2%	QRS,	62.3%	Penicillin	61.4%	Drug dose prednisone	88.4%	Anticoagulant	54.7%	O2 Sat	50.7%
	First generation antipsychotic	62.1%	ALT	55.7%	First generation antihistamine	64.3%	s5 HT4 agonist	62.4%	OrganMetric,	62.8%	Diag Abnormal ECG	88.5%	Macrolide	55.2%	RespRate,	52.1%
	ALT	62.2%	TSH	56.3%	Alpha 2 agonist	64.5%	Beta 1 agonist	62.6%	Tumor Score	63.5%	Tachycardia	88.5%	CPK Score	55.7%	WBC Score,	52.3%
	Corticosteroid	62.5%	Prothrombin	57.4%	Supplement	64.6%	AG Ratio	62.6%	ADV EVT Neuropathy	63.7%	Alpha 2 agonist	90.0%	Corticosteroid	56.3%	WBC	52.8%
	H2 antagonist	62.5%	Neutrohils Band	57.6%	Tetracycline	64.6%	SCr	62.7%	CPK Score	63.8%	H2 antagonist	90.1%	Calcium	56.5%	Immunosuppressant	53.8%
	ADV EVT Neuropathy	62.9%	AG Ratio	57.7%	Tumor Score	64.7%	ADV EVT Sepsis	62.7%	RBC	63.9%	AG Ratio	90.2%	Beta lactam	56.7%	PO2	54.4%
	OrganMetric,	63.0%	SCr	58.0%	s5 HT3 antagonist	64.7%	LDH	62.8%	Tachycardia,	63.9%	HCT	90.8%	Tumor Score	57.0%	Calcium channel blocker	54.6%
	Diag Abnormal ECG	63.1%	Antihypertensive	58.7%	First generation antipsychotic	64.8%	CPK MB	63.0%	Potassium sparing diuretic,	64.1%	ADV EVT Neuropathy	90.8%	Protein	57.4%	HCO3	55.2%
	AG Ratio	63.2%	BNP	58.7%	TSH	64.8%	BNP Score	63.4%	Diag Abnormal ECG	64.1%	First generation antipsychotic	90.9%	SCr	57.5%	LDH Score	55.6%
	Albumin	63.3%	ADV EVT Neuropathy	58.9%	Decongestant	64.8%	LDH Score	63.6%	Electrolyte	64.4%	TSH	91.5%	Tachycardia	57.9%	Tumor Score	55.7%
	Antidiarrheal	63.5%	Albumin	59.0%	Antidiarrheal	64.9%	Basophils	63.7%	Antidiarrheal	64.5%	Prothrombin	91.5%	OrganMetric	57.9%	Phosphate	55.8%
	Decongestant	63.6%	CMRI LVEDV	59.0%	Pressure support	65.2%	Prokinetic	63.9%	First generation antihistamine	64.6%	Analgesia	91.6%	ADV EVT Neuropathy	58.0%	BUN	55.8%
	Potassium sparing diuretic	63.7%	H2 antagonist	59.0%	H2 antagonist,	65.2%	OrganMetric	64.0%			Neutrophils Band	91.6%	WBC	58.3%
							Antidepressant	64.0%					Loop diuretic	58.4%

Abbreviations: CombLoad Myometric, combination myocarditis metric; Comb LabSignal, combination laboratory signal; ADV EVT, adverse event; CombADV EVT, combination adverse event signal; CPK, creatine phosphokinase; CPK-MB, creatine kinase-MB; BNP, B-type natriuretic peptide; AST, aspartate aminotransferase; ALT, alanine aminotransferase; LDH, lactate dehydrogenase; Alk Phos, alkaline phosphate; BPdiast, diastolic blood pressure; BPsys, systolic blood pressure; HCT, hematocrit; SpO2, peripheral O2 saturation; FIO2, fraction of inspired oxygen; PO2, partial pressure of oxygen; HCO3, bicarbonate; LVEDV, left ventricular end diastolic volume; TSH, thyroid stimulating hormone; SCr, serum creatinine; BUN, blood urea nitrogen; AG ratio, albumin/globulin ratio; Diag abnormal ECG, diagnosis of abnormal ECG.

Discussion

Neural Net models based on electronic health records data yield large amounts of time and magnitude data elements. We leveraged the robust monitoring practices of critical care units to assemble a comprehensive model of each patients’ ICI-myocarditis onset, treatments, responses and outcomes. Additionally, we were able to circumvent challenges associated with modeling binary outcomes using convolutional NNs by converting our variables to clinical severity scoring systems, as applied to lab tests, all symptoms and concurrent irAEs. The model permitted complex investigations regarding the interrelationships between inputs and outputs. For example, we used troponin score to investigate the drivers of changes in troponin concentration, demonstrating the impact of immune activation on cardiomyocytes. Here, our analysis showed that across outcomes, troponin and myocarditis severity score were highly correlated with one another and with critical outcomes such as death.

Compared to conventional case report collection, we were able to utilize medical records data to construct a time and magnitude-related data set driven by the clinical time course of myocarditis. Over 300 different parameters were collected per patient and follow-up evaluation continued for outcomes for up to 6 months. Large Phase 3 studies typically collect about 500 data elements per patient, and a clinical study of 350 patients may yield a similar amount of data as we were able to collect on 23 patients. In the case of ICI-related myocarditis, most phase 3 case report forms did not report serial cardiac biomarkers and cardiac imaging results. Having many datapoints focused on a rare event is an advantage, in that fewer, more highly sampled patients can yield more insight into time and magnitude-related drivers of rare events in comparison to many sparsely sampled patients. The collection of large amounts of data on informative patients and analysis using a longitudinal NN modeling approach enables more precise analysis of important co-variates that may otherwise be overlooked. Specifically, the NN modeling approach is able to model non-linear correlations between time-related data arrays and clinical outcomes, which is an extraordinarily difficult task for conventional statistical subgroup analyses. By quantifying the time-related changes in the magnitude of the variables, we were able to model ICI-related myocarditis, including the response to steroid treatment, and the progressive changes in biomarkers of organ physiology as the condition worsened or abated.

We constructed models of clinical outcomes using both scored and unscored data points, and then algorithmically determined the most related input-output relationship for each variable. Here, scoring patients on a 0–5 scale simplified illustration, but did not appear to significantly influence the associations between variables and outputs of interest. Thus, the scoring of laboratory biomarkers and clinical disease severity on a 5-point scale allows a robust, yet convenient means of grouping data along a common time and magnitude axis. Scoring is particularly advantageous when variables are distributed non-linearly, such as troponin elevation ranging from 0.06 to >65 ng/mL. Scoring thus allows us to leverage our existing clinical knowledge to assist in constructing better, and more interpretable models, even though the model reaches the same conclusion without the graphical output-friendly transformation step.

Using this approach, any input of interest could also be used as an output and a model could be constructed to evaluate drivers within the same dataset. For example, each organ system that is affected by drug-related injury has a global marker, which could be used to build clinical models by designating these effects as outputs. Importantly, we severity scored each of these drug-related injury endpoints when collected. Thus, using a similar method, this same database could be used to further investigate colitis, neurological sequelae, thyroiditis, ophthalmological effects, hepatitis and pneumonitis, all of which were observed and scored in these patients. A potential next step in this work is co-modeling of clinical trial data with real-world data from electronic health records. The approach shown here models all the affected organ systems separately and together in the same time frame.

The time and magnitude modeling applied to these patients suggests that there may be a prodromal immune activation as an early signal of immune-related adverse events (irAEs). In these patients, prodromal immune activation leads to systemic organ involvement, and among a tiny fraction of activated patients, can precede the development of severe myocarditis upon progression. While the prediction of ICI-related myocarditis based on clinical risk factors has not been successful, our modeling demonstrated a means to determine factors associated with the early development of myocarditis. This method could potentially be applied to facilitate earlier detection and earlier treatment when biomarkers reveal off target organ involvement as the result of ICI treatment.

In general, the predominant absent parameter in those who died was early and aggressive use of steroid therapy, which differentiates the 11 cases who survived from the 4 cases who did not. In patients who died of fatal ICI-related myocarditis, steroids were used after troponin was very elevated and the myocarditis was likely extensive and ongoing in the heart. These cases also had conduction defects on EKG, and evidence of major secondary organ compromise such as hypotension, tissue hypoxia, compromised cardiac output and cardiogenic shock, and end organ damage.

Severe ICI-related myocarditis is rare and there have not been any baseline pre-treatment variables that identify those who will develop this rare complication, although higher LDH at baseline appears to be associated with this risk. A striking observation in severe cases was the rapid onset of ICI myocarditis in those afflicted, usually apparent in the second week after the first dose. This observation justifies the use of early detection strategies, such as weekly troponin monitoring. Early assessment of ICI-treated patients for symptoms of clinically significant inflammatory activation, such as fever, rash, fatigue, and lack of appetite, justifies an immediate measurement of troponin in these patients. The reason for early detection is clear – patients with rapid onset who receive high doses of steroids before they begin to show heart block, have a better chance of survival.47

Limitations

Our model development was limited in scope by a relatively small number of cases. Hence, the focus of our analysis was to use NN modeling to investigate complex and non-linear associations between time- and magnitude-related changes between hundreds of inputs and only a few outcomes of interest. Although the LOO modeling showed these predictions to be quite stable among the remaining populations, the generalizability of the associations could be further strengthened with a larger sample size and more heterogeneity in the population. Future work in this area will endeavor to develop a predictive NN model for the early detection of ICI-related myocarditis with training and validation. Further investigations should also focus on determining optimal treatments and the ideal timing and dose of those treatments.

Conclusion

Our study shows that a supervised learning NN based on intensive sampling before, during and after irAEs can be used to model rare events such as ICI-related myocarditis and thus provide clinical insight into time and magnitude drivers of progression and treatment outcomes. This approach has the potential to distinguish between likely ICI-related myocarditis and other potential explanations for elevated troponin. These findings direct attention to early detection biomarkers and clinical symptoms as the best means of implementing early and potentially life-saving high dose steroid treatment.

Funding Statement

The NN modeling and analysis of data was funded by Bristol Myers Squibb, and all authors acknowledge that funding supported this study. The funding entity did not provide cases, direct the work or influence the work product in any way.

Abbreviations

MMSE, minimum mean squared error; NN, neural net; irAE, immune system related adverse events; ICI, immune checkpoint inhibitors; CTCAE, common terminology criteria for adverse events; CTLA-4, anti-cytotoxic T lymphocyte-associated antigen; PD-L1, anti-programmed cell death ligand-1; PD-1, anti-programmed cell death 1; NYHA, New York Heart Association; LOO, leave one out; ALT, alanine aminotransferase; AST, aspartate transaminase; LDH, lactate dehydrogenase; CPK, creatine phosphokinase; BNP, B-type natriuretic peptide; CPKMB, creatine kinase-MB; EKG, electrocardiogram.

Data Sharing Statement

All other relevant data are available from the corresponding author upon reasonable request.

Ethics Approval

All clinical procedures and protocols conformed to institutional guidelines and were approved by the institutional review board at the Roswell Park Comprehensive Cancer Center and at University at Buffalo. Due to the retrospective nature of the research and high mortality associated with the condition, a consent waiver was obtained, as well as a HIPPA waiver for individually identifiable information that could not be anonymized such as lab dates.

Consent for Publication

All authors have reviewed this manuscript and have agreed to its content for publication.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

Dr Andres Gomez-Caminero is an employee of Bristol Myers Squibb. Dr Igor Puzanov reports he has received consulting fees from Iovance, Nektar, Oncorus, Merck in past 2 years. Dr Jerome J Schentag reports grants from Bristol Myers Squibb, during the conduct of the study. The authors acknowledge Bristol Myers Squibb funded this work but have no other competing interests.