Question Generation by Transformers
Table 2: Select SQuAD and model-generated questions with various word error rates (WERs). The model-generated questions have been de-tokenized using BERT’s pre-trained WordPiece model.
['[BOLD] SQuAD', '[BOLD] Model-generated', '[BOLD] WER']
The structure of the questions generated by the model are simpler and less detailed than those from SQuAD, but most are nevertheless plausible questions that are grammatically correct. The model-generated questions are relevant to the topic of the context passage and has the correct type of asking words to start the question. For example, although the target question from SQuAD is “by what means were scientists able to liquefy air?”, the model can generate “what is the name of the process of water?”. The questions have a WER of 9 and both can be interpreted as referring to the transformation process of water; more specifically, the condensation process. The model may have not sufficiently learned about the term liquefy but was still able to ask a similar question given limitations in the dataset used for training.