Adapting a FrameNet Semantic Parser for Spoken Language Understanding using Adversarial Learning
Table 2: F-measure (Fmax) on Frame and Argument Identification using (biGRU) and different training datasets.
['[EMPTY]', 'Frame Ident PODCAST', 'Frame Ident PODCAST', 'Argument Ident PODCAST', 'Argument Ident PODCAST']
We observe that simply learning a model on the CALOR textual corpus and applying it on the speech corpus yields very low performances. In PODCAST, for the Frame Identification phase, the parser fails to analyse idioms and oral expressions where the LU should not trigger Frames. Training a parser on the small PODCAST-GOLD or PODCAST-ASR corpus fixes this problem increasing the FI score from 79.6% to 86.3% on manual transcripts and from 79.1% to 86.3% on automatic transcripts. However, the PODCAST corpus is too small to train a model for the Argument Identification task. For this task the only models that give acceptable performances are those that were trained using data from both CALOR and PODCAST. In this experiment we compare our initial biGRU model with a model trained using adversarial domain adaptation. However, since the adaptation technique behaves similarly in all these configurations, we present the results for the most common setup, which consists in training a model on all data sources (CALOR and PODCAST(BOTH)). Under this setup we trained our adversarial model (biGRU+adv) with a ”domain” classifier that distinguishes between two sources, determining if samples come either from CALOR or PODCAST(BOTH). In earlier experiments we tested different domain classification tasks varying the classes, and using unsupervised inferred domains, but the simple 2 domain task yields the best results on PODCAST-ASR.