Table 1.
Technologies and evaluation methods for main CA applications: Part A.
| Personal Assistants and Open-Domain CAs | |||
|---|---|---|---|
| CA | Short Description | Main Technology | Evaluation Method |
| ALICE [48] | a general-purpose chatbot | AIML, | the most human computer |
| pattern matching | winner, 2000, 2001, 2004 | ||
| LSA-bot [50] | ad-hoc implementation | Latent Semantic Analysis | - |
| of the LSA framework | (LSA) | ||
| IRIS [51] | example-based | vector space model | success and |
| chatbot | cosine similarity metric | failure examples | |
| DeepProbe [129] | an open-domain chatbot | seq-2-seq | AUC scores |
| chatbot | |||
| RubyStar [130] | an open-domain chatbot | seq-2-seq, topic detection, | human evaluation |
| engagement monitoring, | by the Alexa Prize | ||
| context tracking | evaluation | ||
| Siri [1] | Apple’s | CNN, | commercial |
| virtual assistant | LSTM | application | |
| Cortana [3] | voice-controlled assistant | NLP, Tellme Networks, | commercial |
| for Microsoft windows | Semantic search database | application | |
| Alexa [23] | Amazon voice assistant | NLP, LSTM | commercial |
| application | |||
| KBot [135] | knowledge | SVM + analytical | F-score, precision, |
| chatbot | queries engine | recall, intent classification | |
| MILABOT [74] | speech/text CA | DRL | Amazon Alexa |
| Prize competition | |||
| Discussion-Bot [154] | question-answering | semantically related | human judges classified |
| chatbot | matching, TF-IDF metric | the answers quality | |
| Goal-Oriented CAs | |||
| CA | Short Description | Main Technology | Evaluation Method |
| SUGILITE [133] | Programming-by-demonstration | frame-based | a lab study: |
| system | dialogue management | task completion time | |
| Safebot [134] | collaborative chatbot | parser+Word2Vec | users’ engagement |
| LIA [55] | learning by | uses combinatory categorial | speed of task |
| instructions agent | grammar (CCG) parser | completeness | |
| CAs for Social Support | |||
| CA | Short Description | Main Technology | Evaluation Method |
| ELIZA [19] | the first CA: | pattern matching | people experience |
| emulates a psychologist | |||
| XiaoIce [107] | a popular social CA | IQ + EQ + Personality | human rating |
| Meena [2] | a sensible chatbot | generative chatbot | human evaluation metric |
| trained end-to-end on | called Sensibleness and | ||
| social media conversations | Specificity Average (SSA) | ||