Algorithm 1 Gradient Boosting Algorithm

Inputs: (i) $\{x_i,y_i\}$, (ii) loss function, $\mathcal{L}$$(y,F(x\left)\right)$, (iii) No. of trees M

Procedure:

(1) Initialize Model with Constant Value $$F_0\left(x\right)=argmi{n}_{\gamma }\left(\sum _{i=1}^n\mathcal{L}(y,\gamma )\right)$$ (1)

(2) Iterate $m=1$ to M

(i) Compute Pseudo-residuals $${\gamma }_{im}={\left[\frac{\partial \mathcal{L}\left({\mathit{y}}_{\mathit{i}}\mathit{F}\left({\mathit{x}}_{\mathit{i}}\right)\right)}{\partial F\left(x_i\right)}\right]}_{F\left(x\right)=F_{m-1}\left(x\right)}^{i=1,...,n}$$ (2)

(ii) Fit a Base-Learner $h_m\left(x\right)$, input $\left\{(x_i,{\gamma }_{im})\right\}$

(iii) $${\gamma }_m=argmi{n}_{\gamma }\sum _{i=1}^n\mathcal{L}({\mathit{y}}_{\mathit{i}},{\mathit{F}}_{\mathit{m}-\mathit{1}}\left({\mathit{x}}_{\mathit{i}}\right)+fl)$$ (3)

(iv) Update Model, $$F_m\left(x\right)=F_{m-1}\left(x\right)+{\gamma }_m{h}_m\left(x\right)$$ (4)