How Transferable are Neural Networks in NLP Applications?
Table 3: Main results of neural transfer learning by INIT. We report test accuracies (%) in this table. E: embedding layer; H: hidden layers; O: output layer. \XBox: Word embeddings are pretrained by word2vec; □: Parameters are randomly initialized); : Parameters are transferred but frozen; : Parameters are transferred and fine-tuned. Notice that the EHO and EHO settings are inapplicable to IMDB→QC and SNLI→MSRP, because the output targets do not share same meanings and numbers of target classes.
['[BOLD] Experiment I Setting', '[BOLD] Experiment I IMDB→MR', '[BOLD] Experiment I IMDB→QC']
A quick observation is that, in both groups, transfer learning of semantically equivalent tasks (IMDB→MR, SNLI→SICK) appears to be successful with an improvement of ∼6%. The results are not surprising and also reported in \newciteNLI.