KERMIT: Generative Insertion-Based Modeling for Sequences
Table 1: The KERMIT architecture works well for three categories of tasks: machine translation, representation learning, and zero-shot cloze question answering. aVaswani et al. (2017) bRadford et al. (2018) cDevlin et al. (2019)
['[EMPTY]', '[BOLD] Machine Translation', '[BOLD] Representation Learning', '[BOLD] Cloze Question Answering']
We apply KERMIT to a diverse set of tasks, finding that our unified approach is capable of matching or exceeding the performance of dedicated state-of-the-art systems without the need for problem-specific components. We first apply KERMIT to machine translation, where the inputs and outputs are parallel sentence pairs. Finally, we apply KERMIT to a zero-shot cloze question-answering task demonstrating the infilling capabilities of the model.