Learning Sparse Sharing Architectures for Multiple Tasks
Table 6: Results of the synthetic experiment. The performance deterioration due to negative transfer is in gray.
['Single task', 'NER 71.05', 'Δ -', 'PP 99.21', 'Δ -']
We employ a 1-layer BiLSTM with 50-dimensional hidden size as shared module and a fully-connected layer as task-specific layer. Our word embeddings are in 50 dimensions and randomly initialized. Note that Δ is defined as the increase of performance compared with single-task models. The synthetic experiment shows that hard sharing suffers from negative transfer, while sparse sharing does not.