Algorithm 1.

LinUCB (Li et al., 2010)

0: Inputs: α ∈ ℝ+

1: for t = 1, 2, 3, …, T do

2:      Observe features of all arms a ∈ $\mathcal{A}$t: xt,a ∈ ℝd

3:      for all a ∈ $\mathcal{A}$t do

4:         if a is new then

5:            Aa ← Id (d-dimensional identity matrix)

6:            ba ← 0d×1 (d-dimensional zero vector)

7:         end if

8:         ${\widehat{\theta }}_a$ ← $A_a^{-1}$ ba

8:         Pt,a ← ${\widehat{\theta }}_a^T$ xt,a + α $\sqrt{x_{t,a}^T{A}_a^{-1}{x}_{t,a}}$

10:    end for

11:    Choose arm at = arg ${max}_{a\in {\mathcal{A}}_{\text{t}}}$ Pt,a with ties broken arbitrarily and observe a real valued payoff rt

12:    Aat ← Aat + xt,at $x_{t,a_t}^T$

13:    bat ← bat + rt xt,at

14:  end for