Inducing Constituency Trees through Neural Machine Translation
Table 4: F1 score with respect to trivial left-branching (LB), right-branching (RB) baselines, and the ground truth parses from Stanford parser (GT) on the corresponding IWSLT test set. We also report accuracy in identifying constituents labeled as ADJP, NP, or PP by the parser. We choose the word-based PRPN model and the word-based ON-LSTM model with the median F1 for each language pair. Bold marks the best performance of each column on the corresponding dataset.
['[BOLD] Model', '[BOLD] F1 (Target) w.r.t  [BOLD] LB', '[BOLD] F1 (Target) w.r.t  [BOLD] RB', '[BOLD] F1 (Target) w.r.t  [BOLD] GT', '[BOLD] Accuracy on  [BOLD] ADJP', '[BOLD] Accuracy on  [BOLD] NP', '[BOLD] Accuracy on  [BOLD] PP']
According to our qualitative analysis, PRPN and ON-LSTM trained with NMT objective are good at discovering noun phrases and prepositional phrases We also notice that the Stanford parser wrongly group ”neutral” and ”tree” first in “a climate neutral tree”, while PRPN correctly groups ”climate neutral” correctly. We also find that both PRPN and ON-LSTM decoders tend to group the BPE subwords under the same constituents in many cases. We observe that PRPN correctly group “chemo@@ syn@@ thesis”, “gre@@ en-@@ light”, etc. (Figure LABEL: fig: bpetrees in the appendix). Although ON-LSTM fails to correctly group the BPE subwords into words in these examples, the subwords are still put under the same intermediate level constituent such as “( the ( ris@@ (ing billion ) ) )”.