Table 1.

Definitions, Meanings, and Applications of Information Theory Concepts to the Study of the Immune System and Immuno-Oncology^b

Concept	Mathematical definition	Meaning	Application	Refs
Entropy, maximum entropy	H(X) = − ∑x∈XP(x) log P(x)	System organization/disorganization	T-cell receptor diversity Information capacity EGFR/Akt signaling Carcinogenesis	[28] [35] [29] [28–30]
Mutual information	I(X;Y) = H(X) − H(X|Y) $H\left(X\mid Y\right)=-\sum _{x\in X,y\in Y}P\left(x,y\right)\text{ log}\frac{P\left(x,y\right)}{P\left(y\right)}.$	Information shared between variables	Biomarker cellular patterns T-cell activation and spatial organization in lymph nodes Cytokines and protein interaction networks Machine learning	[31] [32] [33,34], [36] [44]
Cross and relative entropy	$$\begin{gathered} CE\left(P\Vert Q\right)={\sum }_{x\in X}P\left(x\right)\text{log}\frac{P\left(x\right)}{Q\left(x\right)} \\ -{\sum }_{x\in X}P\left(x\right)\text{ log }P\left(x\right) \end{gathered}$$	Information shared between distributions of variables	Biomarker identification Comparison of transcriptional states (e.g., CD8+ vs. CD4+) Machine learning	[40] [42] [60–63] [41]
Channel capacitya	$C=\underset{P_X(x)}{\text{sup}}I\left(X;Y\right)$	Maximum rates at which information can be reliably transmitted over a communication channel	JAK/STAT signaling Cytokine signaling, gene expression Intrinsic and extrinsic noise in signaling	[34] [16,53] [54,58]
Information transfer and flowb	$\frac{d\mathit{\text{x}}}{dt}=\mathit{\text{F}}(x,t);\frac{dH}{dt}=E(\nabla \cdot \mathit{\text{F}})$	Transfer of information between correlated or uncorrelated variables over time	Spatial and temporal dynamics Information flow in dynamic systems	[60–63] [73]

^aSup is the supremum, or least upper bound of mutual information I(X;Y) over the marginal distribution P_X(x).

^bF is a vector field of a dynamical system (dx/dt), dH/dt is the evolution of entropy (H), equal to the expectation (E) of the divergence (∇) of the vector field F.