Algorithm 1 MDP framework of Eir
for epoch = 1,M do
for g, t in query list do
Query the search engine with g and t.
Update and send state (s, r) to agent
Get action a from agent
while a is not “stop” do
if a is “select” then
Update (s, r) with selection
else if a is “reject” then
Update (s, r) with rejection
else
Translate texts into sequence of concept embeddings.
Relation classification with Bidirectional LSTM
Update (s, r) with classified relation
end if
Send state (s, r) to agent
Get action a from agent
end while
end for
end for