Hierarchical Chatbot

The chatbot has a hierarchical framework comprising the utterance encoder, the context encoder, and the decoder. RNN, as the utterance encoder, encodes the words in an utterance. The transformer encoder, as the context encoder, encodes the utterance encoder’s output, and the RNN decoder takes the attention calculation result of the context encoder as the input.

The strength of such a model is that the utterance encoder makes it possible for us to input a more extended context of the dialogue for restriction by the Transformer’s input size. Considering the Transformer we used isn’t pretrained in advance, the model performance is limited.

The model implemented by us is 0.03 higher than the baseline on F1_BLEU.