Skip to main content
. Author manuscript; available in PMC: 2018 Apr 1.
Published in final edited form as: Inf Sci (N Y). 2016 Aug 16;384:298–313. doi: 10.1016/j.ins.2016.08.038

Table 2.

Learning Rules

Algorithm Learning Rules
Contrastive Divergence ΔWij=∑t(〈vi,thj,t〉d−〈vi,thj,t〉r)
Δai=∑t(〈vi,t〉d−〈vi,t〉r);Δbj=∑t(〈hj,t〉d−〈hj,t〉r)
ΔAifu,t−k=∑t(〈vi,tHfu,t−k〉d−〈vi,tHfu,t−k〉r)
ΔBjfu,t−k=∑t(〈hj,tHfu,t−k〉d−〈hf,tHfu,t−k〉r)
Δβi=∑t(〈vi,t〉d−〈vi,t〉r)ηtu;Δβj=∑t(〈hj,t〉d−〈hj,t〉r)ηtu
Back-Propagation ∂C(θ)∂sj=−∑t(yt−y^t)hj;∂C(θ)∂c=−∑t(yt−y^t)
∂C(θ)∂Wij=−∑t(yt−y^t)sjhj(1−hj)vi
∂C(θ)∂ai=−∑t(yt−y^t)sjhj(1−hj)Wij
∂C(θ)∂bj=−∑t(yt−y^t)sjhj(1−hj)
∂C(θ)∂Aifu,t−k=−∑t(yt−y^t)sjhj(1−hj)WijHfu,t−k
∂C(θ)∂Bjfu,t−k=−∑t(yt−y^t)sjhj(1−hj)Hfu,t−k
∂C(θ)∂βi=−∑t(yt−y^t)sjhj(1−hj)Wijηtu
∂C(θ)∂bj=−∑t(yt−y^t)sjhj(1−hj)ηtu