The Natural Language Decathlon:Multitask Learning as Question Answering
Table 2: Validation metrics for decaNLP baselines: sequence-to-sequence (S2S) with self-attentive transformer layers (w/SAtt), the addition of coattention (+CAtt) over a split context and question, and a question pointer (+QPtr). The last model is equivalent to MQAN. Multitask models use a round-robin batch-level sampling strategy to jointly train on the full decaNLP. The last column includes an additional anti-curriculum (+ACurr) phase that trains on SQuAD alone before switching to the fully joint strategy. Entries marked with ’-’ would correspond to decaScores for aggregates of separately trained models; this is not well-defined without a mechanism for choosing between models.
['5pt.5pt. Dataset', 'Single-task Training S2S', 'Single-task Training w/SAtt', 'Single-task Training +CAtt', 'Single-task Training +QPtr', 'Multitask Training S2S', 'Multitask Training w/SAtt', 'Multitask Training +CAtt', 'Multitask Training +QPtr', '+ACurr']
In our framework, training examples are (question, context, answer) triplets. Our first baseline is a pointer-generator sequence-to-sequence (S2S) model S2S models take in only a single input sequence, so we concatenate the context and question for this model.