Meta-Embedding as Auxiliary Task Regularization
Table 4: Validation (left) & Test (right) Accuracy (%): NER, UDPoS & Sentiment Analysis (only test acc.)
['[BOLD] Task  [BOLD] NER', '[BOLD] Model', '[BOLD] Skipgram', '[BOLD] Skipgram', '[BOLD] FastText', '[BOLD] FastText', '[BOLD] GloVe', '[BOLD] GloVe', '[BOLD] LexVec', '[BOLD] LexVec', '[BOLD] Meta', '[BOLD] Meta']
Based on the validation performance we set λ=0.1 for NER and UDPoS and λ=0.15 for Sentiment Analysis. Unlike the intrinsic tasks, we found a 2-hidden layer AE improved sentence-level meta embedding reconstruction for Sentiment Analysis. This is the only sentence classification dataset and therefore we use sentence-level meta-embedding regularization as opposed to word-level meta-embeddings that are used for intrinsic tasks and the remaining extrinsic tasks (NER and UDPoS tagging).