TABLE 2.

Entropy values for the two entropy versions of the aXb grammar.

Low entropy	High entropy
H[begin-a] = H[3] = [−Σ[0.333*log0.333] = 1.58 H[aX] = H[18] = 4.17 H[Xb] = H[18] = 4.17 H[b-end] = H[3] = 1.58 H[begin-aX] = H[18] = 4.17 H[aXb] = H[Xb-end] = H[18] = 4.17 H[bigram] = 2.86 H[trigram] = 4.17 H[total] = $\frac{H\left[\text{bigram}\right]+H[\text{trigram}]}{2}$ = 3.52	H[begin-a] = H[3] = −Σ[0.333*log0.333] = 1.58 H[aX] = H[54] = 5.75 H[Xb] = H[54] = 5.75 H[b-end] = H[3] = 1.58 H[begin-aX] = H[54] = 5.75 H[aXb] = H[Xb-end] = H[54] = 5.75 H[bigram] = 3.67 H[trigram] = 5.75 H[total] = $\frac{H\left[\text{bigram}\right]+H[\text{trigram}]}{2}$ = 4.71