Parsing Speech: A Neural Approach to Integrating Lexical and Acoustic-Prosodic Information
Table 1: Scores of text-only models on the dev set: 2044 fluent and 3725 disfluent sentences. C-attn denotes content-only attention; CL-attn denotes content+location attention.
['Model', 'F1', 'flat-F1', 'fluent', 'disf']
We first show our results on the model using only text (i.e. xi=ei) to establish a strong baseline, on top of which we can add acoustic-prosodic features. We experiment with the content-only attention model used by \newciteVinyals2015 and the content+location attention of \newciteChorowski2015. For comparison with previous non-neural models, we use a high-quality latent-variable parser, the Berkeley parser Petrov et al. In terms of F1, the content+location attention beats the Berkeley parser by about 2.5% and content-only attention by about 4.5%. Flat-F1 scores for both encoder-decoder models is lower than their corresponding F1 scores, suggesting that the encoder-decoder models do well on predicting the internal structure of EDIT nodes while the reverse is true for the Berkeley parser. It is clear that most of the gains for content+location attention are from disfluent sentences. A possible explanation is the presence of duplicate words or phrases in disfluent sentences, which can be problematic for a content-only attention model. Since our best model is the content+location attention model, we will henceforth refer to it as the “CL-attn” text-only model. All models using acoustic-prosodic features are extensions of this model, which provides a strong text-only baseline. One major challenge of parsing conversational speech is the presence of disfluencies, which are grammatical and prosodic interruptions. Disfluencies include repetitions (‘I am + I am’), repairs (‘I am + we are’), and restarts (‘ What I + Today is the…’), where the ‘+’ corresponds to an interruption point. Repairs often involve parallel grammatical constructions, but they can be more complex, involving hedging, clarifications, etc. However, more explicit modeling of disfluency pattern match characteristics in a dependency parser Honnibal and Johnson (F = 84.1 vs. 76.7 for our text only model). Pattern match features also benefit a neural model for disfluency detection alone (F = 87.0) Zayats et al. F = 87.5) Wang et al. Experiments with oracle disfluencies as features improve the CL-attn text-only parsing performance from 87.85 to 89.38 on the test set, showing that more accurate disfluency modeling is a potential area of improvement.