FIG. 6.

The effect of noise and the input data size on mutual information between two variables. Plots (a–c) illustrate the analysis of mutual information (MI) for three selected pairs, 〈X₁₇, X₁₉〉, 〈X₁₂, X₁₉〉, and 〈X₁₁, X₁₉〉. The first point of each plot shows an MI value computed on the entire set of 5,000 samples with no noise. Every other point shows MI computed on small randomly selected subsets with noise. For each size and noise level, a subset was selected 10 times, and for each subset we added noise and computed MI 100 times. The plots show the average and standard deviation of MI for subset sizes and 19 noise levels (from 5% to 50%). Plot (d) illustrates the same analysis for all other pairs. Each point in (d) is computed similarly to (a–c) and averaged across all pairs, and each error bar is a maximum standard deviation across all pairs.